Recommendation Systems Research

"I Like It, I Like It Not": Noise in User Ratings

Abstract

Many state-of-the-art recommender systems rely on user ratings as ground truth of user taste. However, users are inconsistent in giving feedback, introducing noise that challenges this assumption. In this project, we analyze and characterize the noise in user feedback through movie ratings. We present a user study aimed at quantifying inconsistency-driven noise in user ratings, measuring RMSE values that range from 0.557 to 0.816. We also analyze how factors such as item sorting and time of rating affect this noise.

User interface used in the rating study.

Publications

"I Like It, I Like It Not": Evaluating User Ratings Noise in Recommender Systems Amatriain, X., Pujol, J.M. and Oliver, N. UMAP 2009, Trento, Italy, June 2009.

---

"The Wisdom of the Few": Collaborative Filtering via Expert Opinions

Abstract

Nearest-neighbor collaborative filtering suffers from several shortcomings, including data sparsity and noise, the cold-start problem, and scalability. We present a novel method for recommending items to users based on expert opinions: rather than applying a nearest-neighbor algorithm to the user-rating data, predictions are computed using a set of expert neighbors from an independent dataset, whose opinions are weighted according to their similarity to the user. We validate our approach using the Netflix dataset, measuring both prediction accuracy and recommendation list precision, and report a user study where participants prefer our approach's recommendations.

Publications

"The Wisdom of the Few: A Collaborative Filtering Approach Based on Expert Opinions from the Web" Amatriain, X., Lathia, N., Pujol, J.M., Kwak, H. and Oliver, N. SIGIR 2009, Boston, July 2009.