Collaborative Filtering considered harmful

Much excellent work has been published on collaborative filtering, in particular in terms of recovering missing entries in a matrix. The Netflix contest has contributed a significant amount to the progress in the field.

Alas, reality is not quite as simple as that. Very rarely will we ever be able to query a user about arbitrary movies, books, or other objects. Instead, user ratings are typically expressed as preferences rather than absolute statements: a preference for Die Hard given a generic set of movies only tells us that the user appreciates action movies; however, a preference for Die Hard over Terminator or Rocky suggests that the user might favor Bruce Willis over other action heroes. In other words, the context of user choice is vital when estimating user preferences.

Hence if we attempt to estimate scores $$s_{ui}$$ of user $$u$$ regarding item $$i$$ it is important to use the context within which the ratings have been obtained. For instance, if we are given a session of items $$(i_1, \ldots, i_n)$$ out of which item $$i^*$$ was selected we might want to consider a logistic model of the form:

$$-\log p(i^*|i_1, \ldots, i_n) = \log \left[\sum_{i=1}^n e^{s_{ui}} \right] - s_{ui^*}$$

The option of no action is easy to add, simply by adding the null score $$s_{u0}$$ which captures the event of no action by a user.
Shuang Hong tried out this idea to get a significant performance improvement on a number of collaborative filtering datasets. Bottom line - make sure that the problem you’re solving is actually the one that a) generated the data and b) that will help you in practice. That is, in many cases matrix completion is not the problem you want to solve, even though it might win you benchmarks.

1. canada-news reblogged this from smolix
2. smolix posted this