Building a Recommendation System in Python from Scratch

Quick TL;DR

• A recommendation system predicts what a user is likely to want next — based on their behavior, preferences, or similarity to other users. In Python, recommendation systems are built using collaborative filtering, content-based filtering, or hybrid approaches. 
• This guide covers the core concepts, a step-by-step implementation using pandas and scikit-learn, and practical tips for taking a basic recommender from prototype to production-ready.

Introduction

Every time Netflix suggests a show, Amazon surfaces a product, or Spotify generates a playlist — a recommendation system is working underneath. These systems are no longer exclusive to tech giants. With Python’s data science ecosystem, building a functional recommendation engine from scratch is achievable even without a machine learning background.

This guide starts from zero — no prior ML experience required — and walks through the logic, the math, and the code behind a working recommendation system.

Ready to build real machine learning projects like this and more? Explore HCL GUVI’s Python Course — structured from core fundamentals through data science and ML applications, with hands-on projects, mentorship, and placement support built in.

What Is a Recommendation System?

A recommendation system is an algorithm that filters information to predict what a specific user would prefer from a larger set of options. It answers one question — given what I know about this user, what should I show them next?

There are three primary approaches:

• Collaborative Filtering — recommends based on what similar users liked. If User A and User B both liked the same five movies, and User B liked a sixth, that sixth gets recommended to User A.
• Content-Based Filtering — recommends based on item attributes. If a user liked an action movie with a specific director, the system surfaces other action movies by the same director.
• Hybrid Systems — combine both approaches to compensate for each method’s weaknesses.

For beginners, collaborative filtering is the clearest starting point — it requires no item metadata, only user interaction data.

Read More: Top 10 Python Projects for Beginners with Source Code

Setting Up the Environment

Install the required libraries before writing any code:

pip install pandas numpy scikit-learn

The full implementation in this guide uses only these three libraries — no specialized ML frameworks required.

The Dataset

For this guide, a simple user-movie ratings matrix is used — the same structure behind most real-world recommendation engines:

This creates a long-format dataframe — each row is one user rating one movie. The next step is converting this into a matrix format the algorithm can process.

Ready to build real machine learning projects like this and more? Explore HCL GUVI’s Python Course — structured from core fundamentals through data science and ML applications, with hands-on projects, mentorship, and placement support built in.

Building the User-Item Matrix

The user-item matrix places users on one axis and items on the other — each cell holds the rating that user gave that item. Empty cells represent movies a user has not rated yet — these are the gaps the recommendation system fills.

Output:

Zeros represent unrated movies — not actual zero ratings. This distinction matters when interpreting model output.

Generating Recommendations

With similarity scores in place, recommendations are generated by — finding the most similar users to the target user, identifying movies those users rated highly that the target user has not seen, and weighting those ratings by similarity score:]

Output:

Bob has not seen Interstellar or Dunkirk — based on his similarity to Alice and Carol, the system recommends Interstellar first.

Common Mistakes Beginners Make

1. Treating zeros as actual ratings — Unfilled cells in the user-item matrix represent missing data, not zero preference. Filling with zeros is a pragmatic workaround — but more advanced implementations use matrix factorization to handle sparsity properly.

2. Using too little data — Cosine similarity breaks down with sparse matrices and few users. Real systems need thousands of interactions before similarity scores become meaningful.

3. Never evaluating the model — A recommendation system with no evaluation metric is guesswork. Use RMSE for rating prediction or precision@k for ranking quality to measure whether recommendations are actually improving.

4. Ignoring the cold start problem — New users with no history and new items with no ratings cannot be recommended through collaborative filtering alone. A hybrid approach or onboarding questionnaire is needed to handle cold starts.

5. Skipping normalization — Users rate differently. One user’s 3 is another’s 5. Mean-centering ratings before computing similarity produces more reliable results.

Conclusion

Recommendation systems are one of the highest-impact applications of data science — and Python makes them accessible from day one. Starting with a user-item matrix and cosine similarity covers the core of collaborative filtering without requiring any specialized ML framework. Content-based filtering layers on top cleanly using TF-IDF and the same similarity logic.

The implementation in this guide is deliberately minimal — production systems add evaluation pipelines, matrix factorization, real-time serving, and A/B testing on top. But the logic underneath is exactly what was built here.

Master the fundamentals first — scale comes after.

FAQ

1. What is a recommendation system in Python? 

A recommendation system in Python is an algorithm — typically built with pandas, scikit-learn, or specialized libraries like Surprise — that predicts user preferences and surfaces relevant items based on past behavior or item attributes.

2.What is the difference between collaborative and content-based filtering? 

Collaborative filtering uses patterns across multiple users’ behavior to generate recommendations. Content-based filtering uses the attributes of items themselves. Collaborative filtering requires no item metadata — content-based filtering requires no data from other users.

3.What libraries are used to build recommendation systems in Python? 

The most common are pandas and NumPy for data handling, scikit-learn for similarity computation, and Surprise or LightFM for more advanced matrix factorization approaches.

4.What is the cold start problem in recommendation systems? 

The cold start problem occurs when a new user or new item has no historical data — making it impossible for collaborative filtering to generate meaningful recommendations. Hybrid systems and onboarding flows are the standard solutions.

5.How is recommendation accuracy measured? 

Common metrics include RMSE and MAE for rating prediction accuracy, and precision@k and recall@k for ranking quality — measuring how many of the top-k recommendations were actually relevant to the user.

6.Do I need machine learning experience to build a recommendation system? 

No — the core logic of collaborative filtering is based on similarity math that any Python beginner can implement with pandas and scikit-learn. Advanced techniques like matrix factorization build on these fundamentals.

7.What is cosine similarity in recommendation systems? 

Cosine similarity measures the angle between two vectors in multi-dimensional space — in this context, two users’ rating vectors. A score of 1 means identical preference patterns; 0 means no overlap. It is the standard similarity metric for user-based collaborative filtering.