Improving rating estimation in recommender systems using aggregation- and variance-based hierarchical models

Previous work on using external aggregate rating information showed that this information can be incorporated in several different types of recommender systems and improves their performance. In this paper, we propose a more general class of methods that combine external aggregate information with individual ratings in a novel way. Unlike the previously proposed methods, one of the defining features of this approach is that it takes into the consideration not only the aggregate average ratings but also the variance of the aggregate distribution of ratings. The methods proposed in this paper estimate unknown ratings by finding an optimal linear combination of individual-level and aggregate-level rating estimators in a form of a hierarchical regression (HR) model that is grounded in the theory of statistics and machine learning. The proposed HR model is general enough so that the standard individual-level recommender systems and naive aggregate methods constitute special cases of this model. We show that for the general HR model, the presence of the aggregate variance, surprisingly, does not significantly improve estimation of unknown ratings vis-a-vis the case when only aggregate average ratings are considered. In the paper, we experimentally show that the optimal linear combination approach significantly dominates all other special cases, including the classical non-aggregated case and our previously studied aggregate methods, and therefore is the method of choice.