Dan G gives a good explanation for the kid movie recomendations.

I think Amazon does this much better. Amazon's recommendation algorithm is based on a product level comparison rather than a customer level comparison. For each product they make a list of the related products. The related product lists are pretty good because Amazon has a lot of data at the product level. They then compile the related products lists for the products you purchased to produce your recommendations. If you liked merle haggard and Takashi Miike, maybe you will like Shion Sono and Willie Nelson. This is a pretty simple algorithm.

It also completely avoids the "Beatles Problem". Amazon doesn't check for what other products customers who like both merle haggard and Takashi Miike also like. Who cares? There just aren't enough customers who like both merle haggard and Takashi Miike to get valid data.

Here is an uncharacteristically bad Malcolm Gladwell article from 1999 on collaborative filtering. Gladwell dismisses the Amazon algorithm in favor of the powerful collaborative filtering algorithms that are just around the corner. It is 2005 and they are still just around the corner.

I especially like the quote:

>"What we'd like to see is nice little clusters," Goldberg says. "But, when you look at the results, what you see is something like a cloud with sort of bunches, and nothing that is nicely defined. It's kind of like looking into the night sky. It's very hard to identify the constellations."

Which is to say collaborative filtering doesn't work very well, but Gladwell doesn't pick up on this.

Netflix is a special case. They don't want to just recommend the films that they think you will like. They want to recommend the films that they think you will like and that they have a lot of copies of just sitting around.

They make it hard to rent big blockbuster movies when they first come out. They don't recommend such movies; you have to search for them by name. Netflix wants to keep the number of new movies people rent to a minimum so that they can keep the inventory of each new release as low as possible.

Relatively obscure films cause a similar problem. Criterion collection DVDs don't grow on trees. The wait for some obscure movies can be very long. I suspect Netflix doesn't have very many copies of the more obscure films. I also suspect people who like obscure movies tend to rent a lot of movies and are thus not particularly desirable customers .

Posted by: Joe O on June 2, 2005 4:54 AM