What are Recommendation Systems?

Nearly every time you interact with a modern digital platform, whether streaming a movie, browsing an e-commerce site, or listening to a music service, you are interacting with a recommendation system. These systems are sophisticated information filters designed to predict your preferences and present you with items you are likely to find interesting or useful. Their primary function, especially when dealing with large catalogs of content and products, is to combat information overload and create a personalized experience for each user.

At its core, a recommendation system predicts the "rating" or "preference" a user would assign to an item. This doesn't always mean a 5-star rating. A preference can also be inferred from user behavior, such as watching a full video, purchasing a product, or spending time on an article. The system's main objective is to sift through millions of items and surface a small, relevant subset tailored specifically to an individual's tastes.

How Recommenders Work at a High Level

A recommendation engine operates on two primary sources of information: data about the users and data about the items. It processes this information to learn patterns and relationships. For example, it might learn that users who like movie A also tend to like movie B, or that users who have shown interest in items with specific attributes (like a movie's genre or director) will likely enjoy other items with similar attributes.

The output is typically a ranked list of recommendations, ordered from most to least likely to be of interest to the user. This process forms a continuous feedback loop: the user interacts with the recommendations, generating new data that is then used to refine and improve future suggestions.

A high-level view of a recommendation system's data flow. User and item data feed into the engine, which produces a ranked list for the user, whose interactions generate new feedback data.

The Business Impact

From a business perspective, recommendation systems are powerful tools for driving user engagement and revenue.

• For e-commerce: They can increase sales by suggesting relevant products, leading to higher conversion rates and larger average order values. A classic example is Amazon's "Customers who bought this item also bought" feature.
• For content platforms: They increase user retention by suggesting engaging content, such as movies on Netflix or videos on YouTube. By keeping users on the platform longer, they increase opportunities for advertising or strengthen subscription value.
• For social media: They curate personalized feeds that surface relevant posts and connections, making the user's experience more compelling and encouraging them to return.

Two Primary Recommendation Tasks

The output of a recommendation system can generally be framed in one of two ways, and understanding this distinction is important for building and evaluating them.

1. Rating Prediction: This task involves predicting the specific rating a user might give to an item. For example, the system might predict that a particular user will rate a movie 4.2 out of 5 stars. The goal is to make this prediction as accurate as possible.
2. Item Ranking: This is the more common task, where the system generates an ordered list of the top N items for a user. Here, the exact predicted rating is less important than the relative order of the items. The primary goal is to ensure that the most relevant items appear at the top of the list.

Throughout this course, we will build systems that perform both tasks and learn how to evaluate each one appropriately. With this general understanding of what these systems do, we can now examine the different algorithmic approaches used to build them.