Project - Recommendation Systems: Movie Recommendation System

Marks: 40

Context

Online streaming platforms like Netflix have plenty of movies in their repository and if we can build a Recommendation System to recommend relevant movies to users, based on their historical interactions, this would improve customer satisfaction and hence, it will also improve the revenue of the platform. The techniques that we will learn here will not only be limited to movies, it can be any item for which you want to build a recommendation system.

Objective

In this project we will be building various recommendation systems:

we are going to use the ratings dataset.

Dataset

The ratings dataset contains the following attributes:

Sometimes, the installation of the surprise library, which is used to build recommendation systems, faces issues in Jupyter. To avoid any issues, it is advised to use Google Colab for this case study.

Let's start by mounting the Google drive on Colab.

Installing surprise library

Importing the necessary libraries and overview of the dataset

Loading the data

Let's check the info of the data

Question 1: Exploring the dataset (7 Marks)

Let's explore the dataset and answer some basic data-related questions:

Q 1.1 Print the top 5 rows of the dataset (1 Mark)

Q 1.2 Describe the distribution of ratings. (1 Mark)

**Write your Answer here:1. From the countplot, rating 4 has highest count with >25000(highest count of ratings) followed by rating 3 with count around 20000.

  1. Ratings 5 and 3.5 have similar counts with <5000 counts difference.
  2. Rating 0.5 have least number of rating count.

Q 1.3 What is the total number of unique users and unique movies? (1 Mark)

Write your answer here:__ There are 671 unique users in the dataset.

Write your answer here:__ There are 9066 unique movies in the given dataset.There is a possibility of 671*9066 =6083286 ratings in the dataset.But we have only 100004 i.e not every user rated the movies. We can build a recommendation system to recommend movies to the used they have not interacted at all.

Q 1.4 Is there a movie in which the same user interacted with it more than once? (1 Mark)

Write your Answer here:__Here the sum is equal to the total observations which means there is only one interaction between user and movies and not more than that.

Q 1.5 Which is the most interacted movie in the dataset? (1 Mark)

Write your Answer here:__Movie with movieid=356 has more interactions with 341 users.However there are 330 more users most likely to interact with the movie. We can build a recommendation model to predict the likelihood.

Write your Answer here:__From the graph, we can interpret that this movie with movieid 356 has most 4 rating followed by rating 5. That is users liked the movie most of the time. Ratings 2 and below are less in count which means very few users disliked and have given less rating.

Q 1.6 Which user interacted the most with any movie in the dataset? (1 Mark)

Write your Answer here:___User with id 547 has interacted with the movies with maximum count of 2391. Again, there are still 6675 movies that the user is likely to get interacted with.

Q 1.7 What is the distribution of the user-movie interactions in this dataset? (1 Mark)

Write your Answer here:__The distribution is highly skewed to the right which tells that only few users interacted with more than 100 movies.

As we have now explored the data, let's start building Recommendation systems

Question 2: Create Rank-Based Recommendation System (3 Marks)

Model 1: Rank-Based Recommendation System

Rank-based recommendation systems provide recommendations based on the most popular items. This kind of recommendation system is useful when we have cold start problems. Cold start refers to the issue when we get a new user into the system and the machine is not able to recommend movies to the new user, as the user did not have any historical interactions in the dataset. In those cases, we can use rank-based recommendation system to recommend movies to the new user.

To build the rank-based recommendation system, we take average of all the ratings provided to each movie and then rank them based on their average rating.

Now, let's create a function to find the top n movies for a recommendation based on the average ratings of movies. We can also add a threshold for a minimum number of interactions for a movie to be considered for recommendation.

We can use this function with different n's and minimum interactions to get movies to recommend

Recommending top 5 movies with 50 minimum interactions based on popularity

Recommending top 5 movies with 100 minimum interactions based on popularity

Recommending top 5 movies with 200 minimum interactions based on popularity

Now that we have seen how to apply the Rank-Based Recommendation System, let's apply the Collaborative Filtering Based Recommendation Systems.

Model 2: User based Collaborative Filtering Recommendation System (7 Marks)

collaborative_filtering.PNG

In the above interactions matrix, out of users B and C, which user is most likely to interact with the movie, "The Terminal"?

In this type of recommendation system, we do not need any information about the users or items. We only need user item interaction data to build a collaborative recommendation system. For example -

  1. Ratings provided by users. For example - ratings of books on goodread, movie ratings on imdb etc
  2. Likes of users on different facebook posts, likes on youtube videos
  3. Use/buying of a product by users. For example - buying different items on e-commerce sites
  4. Reading of articles by readers on various blogs

Types of Collaborative Filtering

Building Similarity/Neighborhood based Collaborative Filtering

test_image

Building a baseline user-user similarity based recommendation system

Below we are loading the rating dataset, which is a pandas DataFrame, into a different format called surprise.dataset.DatasetAutoFolds, which is required by this library. To do this, we will be using the classes Reader and Dataset. Finally splitting the data into train and test set.

Making the dataset into surprise dataset and splitting it into train and test set

Build the first baseline similarity based recommendation system using cosine similarity and KNN

Q 3.1 What is the RMSE for baseline user based collaborative filtering recommendation system? (1 Mark)

Write your Answer here:__ From the baseline uder model, RMSE=0.9925. We can use improve the RMSE score by GridSearchCV by tuning the hyperparameters.

Q 3.2 What is the Predicted rating for an user with userId=4 and for movieId=10 and movieId=3? (1 Mark)

Let's us now predict rating for an user with userId=4 and for movieId=10

Write your Answer here:___From the prediction, the estimated user-item pair is 3.62 whereas the actual user-item pair is 4

Let's predict the rating for the same userId=4 but for a movie which this user has not interacted before i.e. movieId=3

Write your Answer here:__Here the user-item pair interaction is predicted as 3.20 whereas originally there was zero user-item pair interaction. We can improve this by tuning the hyperparameters.

Improving user-user similarity based recommendation system by tuning its hyper-parameters

Below we will be tuning hyper-parmeters for the KNNBasic algorithms. Let's try to understand different hyperparameters of KNNBasic algorithm -

For more details please refer the official documentation https://surprise.readthedocs.io/en/stable/knn_inspired.html

Q 3.3 Perform hyperparameter tuning for the baseline user based collaborative filtering recommendation system and find the RMSE for tuned user based collaborative filtering recommendation system? (3 Marks)

Once the grid search is complete, we can get the optimal values for each of those hyperparameters as shown above.

Below we are analysing evaluation metrics - RMSE and MAE at each and every split to analyze the impact of each value of hyperparameters

Now, let's build the final model by using tuned values of the hyperparameters, which we received by using grid search cross-validation.

Write your Answer here:__After tuning the hyperparameters the RMSE reduced to 0.9571 from 0.9925. We have slightly improved the model performance by the tuning.

Q 3.4 What is the Predicted rating for an user with userId =4 and for movieId= 10 and movieId=3 using tuned user based collaborative filtering? (1 Mark)

Let's us now predict rating for an user with userId=4 and for movieId=10 with the optimized model

Write your Answer here:__Comparing baseline model and optimized model, the predicted values are 1. baseline model-3.62 2.Optimized model-3.55

Below we are predicting rating for the same userId=4 but for a movie which this user has not interacted before i.e. movieId=3, by using the optimized model as shown below -

Write your Answer here:__Comparing baseline model and optimized model, the predicted values are 1. baseline model-3.20 2.Optimized model-3.72

Identifying similar users to a given user (nearest neighbors)

We can also find out the similar users to a given user or its nearest neighbors based on this KNNBasic algorithm. Below we are finding 5 most similar user to the userId=4 based on the msd distance metric

Implementing the recommendation algorithm based on optimized KNNBasic model

Below we will be implementing a function where the input parameters are -

Predicted top 5 movies for userId=4 with similarity based recommendation system

Q 3.5 Predict the top 5 movies for userId=4 with similarity based recommendation system (1 Mark)

Model 3: Item based Collaborative Filtering Recommendation System (7 Marks)

Q 4.1 What is the RMSE for baseline item based collaborative filtering recommendation system ?(1 Mark)

Write your Answer here:__The RMSE for the baseline item based model is 1.0032 .We can tune the hyperparameters to reduce it.

Let's us now predict rating for an user with userId=4 and for movieId=10

Q 4.2 What is the Predicted rating for an user with userId =4 and for movieId= 10 and movieId=3? (1 Mark)

Write your Answer here:__The actual rating for the user-item interaction is 4 and the predicted is 3.55 using similarity based model.

Let's predict the rating for the same userId=4 but for a movie which this user has not interacted before i.e. movieId=3

Write your Answer here:__ The predicted user-item interaction is 4.07 whereas the actual user-item interaction is zero.We tune the hyperparameters.

Q 4.3 Perform hyperparameter tuning for the baseline item based collaborative filtering recommendation system and find the RMSE for tuned item based collaborative filtering recommendation system? (3 Marks)

Once the grid search is complete, we can get the optimal values for each of those hyperparameters as shown above

Below we are analysing evaluation metrics - RMSE and MAE at each and every split to analyze the impact of each value of hyperparameters

Now let's build the final model by using tuned values of the hyperparameters which we received by using grid search cross-validation.

Write your Answer here:__By tuning the hyperparameters the RMSE has increased to 0.9433 whereas the baseline model RMSE is 1.0032. Hence the model has not shown improvement by doing GridSearchCV.

Q 4.4 What is the Predicted rating for an item with userId =4 and for movieId= 10 and movieId=3 using tuned item based collaborative filtering? (1 Mark)

Let's us now predict rating for an user with userId=4 and for movieId=10 with the optimized model as shown below

Write your Answer here:___Comparing with the baseline model, the predicted user-item intreraction remains the same with value 3.55.

Let's predict the rating for the same userId=4 but for a movie which this user has not interacted before i.e. movieId=3, by using the optimized model:

Write your Answer here:__The user-item interaction for the optimzed model is 3.87 where it is 4.07 for the baseline model whereas the actual rating is 0.

Identifying similar items to a given item (nearest neighbors)

We can also find out the similar items to a given item or its nearest neighbors based on this KNNBasic algorithm. Below we are finding 5 most similar items to the movieId=3 based on the msd distance metric

Predicted top 5 movies for userId=4 with similarity based recommendation system

Q 4.5 Predict the top 5 movies for userId=4 with similarity based recommendation system (1 Mark)

Model 4: Based Collaborative Filtering - Matrix Factorization using SVD (7 Marks)

Model-based Collaborative Filtering is a personalized recommendation system, the recommendations are based on the past behavior of the user and it is not dependent on any additional information. We use latent features to find recommendations for each user.

Latent Features: The features that are not present in the empirical data but can be inferred from the data. For example:

test_image

Now if we notice the above movies closely:

test_image

Here Action, Romance, Suspense and Comedy are latent features of the corresponding movies. Similarly, we can compute the latent features for users as shown below:

test_image

Singular Value Decomposition (SVD)

SVD is used to compute the latent features from the user-item matrix. But SVD does not work when we miss values in the user-item matrix.

First we need to convert the below movie-rating dataset:

test_image

into an user-item matrix as shown below:

test_image

We have already done this above while computing cosine similarities.

SVD decomposes this above matrix into three separate matrices:

U-matrix

test_image

the above matrix is a n x k matrix, where:

Sigma-matrix

test_image

the above matrix is a k x k matrix, where:

V-transpose matrix

test_image

the above matrix is a kxn matrix, where:

Build a baseline matrix factorization recommendation system

Q 5.1 What is the RMSE for baseline SVD based collaborative filtering recommendation system? (1 Mark)

Write your Answer here:___RMSE for baseline SVD on the testset is 0.9034 which is lower that RMSE for baseline similarlity based model(1.0032) and even lesser than optimized similarity based model(0.9433).

Q 5.2 What is the Predicted rating for an user with userId =4 and for movieId= 10 and movieId=3? (1 Mark)

Let's us now predict rating for an user with userId=4 and for movieId=10

Write your Answer here:__We can see that using matrix factorization model, the predicted user-item pair is 4.15 whereas the actual rating is 4. We have slightly over estimated the rating. We can improve this by tuning the hyperparameters.

Let's predict the rating for the same userId=4 but for a movie which this user has not interacted before i.e. movieId=3:

Write your Answer here:_We have estimated the rating as 3.55 whereas the actual user-item pair rating is zero.

Improving matrix factorization based recommendation system by tuning its hyper-parameters

In SVD, rating is predicted as -

$$\hat{r}_{u i}=\mu+b_{u}+b_{i}+q_{i}^{T} p_{u}$$

If user $u$ is unknown, then the bias $b_{u}$ and the factors $p_{u}$ are assumed to be zero. The same applies for item $i$ with $b_{i}$ and $q_{i}$.

To estimate all the unknown, we minimize the following regularized squared error:

$$\sum_{r_{u i} \in R_{\text {train }}}\left(r_{u i}-\hat{r}_{u i}\right)^{2}+\lambda\left(b_{i}^{2}+b_{u}^{2}+\left\|q_{i}\right\|^{2}+\left\|p_{u}\right\|^{2}\right)$$

The minimization is performed by a very straightforward stochastic gradient descent:

$$\begin{aligned} b_{u} & \leftarrow b_{u}+\gamma\left(e_{u i}-\lambda b_{u}\right) \\ b_{i} & \leftarrow b_{i}+\gamma\left(e_{u i}-\lambda b_{i}\right) \\ p_{u} & \leftarrow p_{u}+\gamma\left(e_{u i} \cdot q_{i}-\lambda p_{u}\right) \\ q_{i} & \leftarrow q_{i}+\gamma\left(e_{u i} \cdot p_{u}-\lambda q_{i}\right) \end{aligned}$$

There are many hyperparameters to tune in this algorithm, you can find a full list of hyperparameters here

Below we will be tuning only three hyperparameters -

Q 5.3 Perform hyperparameter tuning for the baseline SVD based collaborative filtering recommendation system and find the RMSE for tuned SVD based collaborative filtering recommendation system? (3 Marks)

Once the grid search is complete, we can get the optimal values for each of those hyperparameters, as shown above.

Below we are analysing evaluation metrics - RMSE and MAE at each and every split to analyze the impact of each value of hyperparameters

Now, we will the build final model by using tuned values of the hyperparameters, which we received using grid search cross-validation above.

Q 5.4 What is the Predicted rating for an user with userId =4 and for movieId= 10 and movieId=3 using SVD based collaborative filtering? (1 Mark)

Let's us now predict rating for an user with userId=4 and for movieId=10 with the optimized model

Write your Answer here:_Using optimized SVD model, the estimated rating is 3.99 which is close to the actual rating i.e 4.

Let's predict the rating for the same userId=4 but for a movie which this user has not interacted before i.e. movieId=3:

Q 5.5 Predict the top 5 movies for userId=4 with SVD based recommendation system?(1 Mark)

Predicting ratings for already interacted movies

Below we are comparing the rating predictions of users for those movies which has been already watched by an user. This will help us to understand how well are predictions are as compared to the actual ratings provided by users

Here we are comparing the predicted ratings by similarity based recommendation system against actual ratings for userId=7

Write your Answer here:__From the distribution plot,the predicted ratings value are more in around 3 to 4 whereas the actual rating values are discret from 1 to 5. The distribution of predicted values are only between 2.5 and 4.The actual ratings distribution has its peak at 3.

Below we are comparing the predicted ratings by matrix factorization based recommendation system against actual ratings for userId=7

Precision and Recall @ k

RMSE is not the only metric we can use here. We can also examine two fundamental measures, precision and recall. We also add a parameter k which is helpful in understanding problems with multiple rating outputs.

Precision@k - It is the fraction of recommended items that are relevant in top k predictions. Value of k is the number of recommendations to be provided to the user. One can choose a variable number of recommendations to be given to a unique user.

Recall@k - It is the fraction of relevant items that are recommended to the user in top k predictions.

Recall - It is the fraction of actually relevant items that are recommended to the user i.e. if out of 10 relevant movies, 6 are recommended to the user then recall is 0.60. Higher the value of recall better is the model. It is one of the metrics to do the performance assessment of classification models.

Precision - It is the fraction of recommended items that are relevant actually i.e. if out of 10 recommended items, 6 are found relevant by the user then precision is 0.60. The higher the value of precision better is the model. It is one of the metrics to do the performance assessment of classification models.

See the Precision and Recall @ k section of your notebook and follow the instructions to compute various precision/recall values at various values of k.

To know more about precision recall in Recommendation systems refer to these links :

https://surprise.readthedocs.io/en/stable/FAQ.html

https://medium.com/@m_n_malaeb/recall-and-precision-at-k-for-recommender-systems-618483226c54

Question6: Compute the precision and recall, for each of the 6 models, at k = 5 and 10. This is 6 x 2 = 12 numerical values? (4 marks)

Question 7 ( 5 Marks)

7.1 Compare the results from the base line user-user and item-item based models.

7.2 How do these baseline models compare to each other with respect to the tuned user-user and item-item models?

7.3 The matrix factorization model is different from the collaborative filtering models. Briefly describe this difference. Also, compare the RMSE and precision recall for the models.

7.4 Does it improve? Can you offer any reasoning as to why that might be?

Write your Answer here:__

  1. The baseline user-user based recommendation model and item -item based recommendation model performed similar with RMSE 0.9925 and 1.0032. However the optimized models for both user-user(RMSE-0.9571) and item-item(RMSE-0.9433)gave better results when compared to baseline models. 2.Instead of predicting random ratings, Collaborative filtering user user ratings to find similarities and hence the results are better. 3.RMSE for baseline Matrix Factorization is 0.9034 which is much better than Collaborative filtering models(0.9433) 4.Tuning the SVD has improved the RMSE much better as the value of RMSE reduced to 0.8952. 5.SVD uses latent features to build a recommendation model and assumes both the users and items in lower dimensions, we get optimized results compared to Collaborative Filtering.
  2. Collaborative Filtering precision and recall values for k=5 are 0.683 and 0.355 and for SVD 0.748 and 0.384. Clearly SVD performed well in compared to Collaborative filtering considering precision values. But the collaborative filtering works good as it has good recall value.
  3. For K=10, SVD performed well with precision value 0.725 and recall value 0.524.However the recall for Collaborative Filtering has been increased to 0.508 which means this model performance is also good but the precison value has been slightly decreased(0.664). 8.Finally depending on the business requirements whether they want good precision recall or lower RMSE the model will can be choosen.
  4. From RMSE perspective,tuned SVD performed well and can give good recommendations to the user.

Conclusions

In this case study, we saw three different ways of building recommendation systems:

We also understood advantages/disadvantages of these recommendation systems and when to use which kind of recommendation systems. Once we build these recommendation systems, we can use A/B Testing to measure the effectiveness of these systems.

Here is an article explaining how Amazon use A/B Testing to measure effectiveness of its recommendation systems.