Menu

Feature Extraction Using TF-IDF

Feature Extraction Using TF-IDF

Machine learning algorithms cannot interpret plain text directly. Therefore, the combined text must be transformed into numerical vectors before similarity can be calculated.

Code

vectorizer = TfidfVectorizer(
stop_words='english'
)
tfidf_matrix = vectorizer.fit_transform(
news['content']
)

Explanation

The TF-IDF Vectorizer converts each news article into a numerical feature vector based on the importance of the words it contains.

Common English stop words are automatically removed to improve feature quality.

Output

The textual data is converted into a TF-IDF feature matrix.