News Summarization

Problem Context

The project aimed to create a system that could summarize news articles into concise versions (~250 words) and provide personalized recommendations based on user interests. This was an exercise in text processing, feature extraction, and building a basic NLP pipeline.

Collection

News articles were retrieved via public APIs and processed into raw text.

• Automated gathering from a news API endpoint

• Stored headlines, full text, and metadata (e.g., source, category)

Preparation

I cleaned and transformed the text for modeling.

• Removed stopwords, punctuation, and special characters

• Tokenized text and applied TF-IDF feature extraction

• Ensured consistent casing and lemmatization

Baseline

A simple frequency-based extractive summarizer was used as a baseline.

• Selected top-N sentences based on TF-IDF scores

• Produced rough but informative summaries

Modeling

I built a similarity-based recommendation engine on top of the summarizer.

• Cosine similarity between TF-IDF vectors determined “related” articles

• Pipeline combined summarization + recommendation for each article

Evaluation

Since NLP evaluation is less straightforward, I used both quantitative and qualitative checks.

• Metrics: ROUGE scores against human-written summaries

• Manual review of summary quality and recommendation relevance

Refinement

The system was optimized for usability and readability.

• Limited summaries to ~250 words for consistency

• Filtered recommendations by category to improve personalization

Conclusion

The system successfully produced concise summaries (~250 words) and suggested related articles using cosine similarity. While not as sophisticated as abstractive deep learning models, this project demonstrated strong understanding of the NLP pipeline, feature engineering for text, and evaluation methods for summarization and recommendation tasks.