Comments on: Weighted Slope One in Haskell: collaborative filtering in 29 lines of code http://www.serpentine.com/blog/2007/08/27/weighted-slope-one-in-haskell-collaborative-filtering-in-29-lines-of-code/ Bryan O'Sullivan's blog Wed, 08 Feb 2012 06:41:38 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: Functional Programming and Collective Intelligence - IV - Matthew Podwysocki's Blog http://www.serpentine.com/blog/2007/08/27/weighted-slope-one-in-haskell-collaborative-filtering-in-29-lines-of-code/comment-page-1/#comment-222578 Functional Programming and Collective Intelligence - IV - Matthew Podwysocki's Blog Sat, 11 Apr 2009 05:19:14 +0000 http://www.serpentine.com/blog/2007/08/27/weighted-slope-one-in-haskell-collaborative-filtering-in-29-lines-of-code/#comment-222578 [...] this implementation, I’m going to tackle the Weighted Slope One.  Using the Haskell implementation by Bryan O’Sullivan as my source of inspiration.  I find that going between Haskell and F# to be [...] [...] this implementation, I’m going to tackle the Weighted Slope One.  Using the Haskell implementation by Bryan O’Sullivan as my source of inspiration.  I find that going between Haskell and F# to be [...]

]]> By: Functional Programming and Collective Intelligence - IV - Matthew Podwysocki - CodeBetter.Com - Stuff you need to Code Better! http://www.serpentine.com/blog/2007/08/27/weighted-slope-one-in-haskell-collaborative-filtering-in-29-lines-of-code/comment-page-1/#comment-222577 Functional Programming and Collective Intelligence - IV - Matthew Podwysocki - CodeBetter.Com - Stuff you need to Code Better! Sat, 11 Apr 2009 05:11:45 +0000 http://www.serpentine.com/blog/2007/08/27/weighted-slope-one-in-haskell-collaborative-filtering-in-29-lines-of-code/#comment-222577 [...] this implementation, I’m going to tackle the Weighted Slope One.  Using the Haskell implementation by Bryan O’Sullivan as my source of inspiration.  I find that going between Haskell and F# to be [...] [...] this implementation, I’m going to tackle the Weighted Slope One.  Using the Haskell implementation by Bryan O’Sullivan as my source of inspiration.  I find that going between Haskell and F# to be [...]

]]> By: Chris K http://www.serpentine.com/blog/2007/08/27/weighted-slope-one-in-haskell-collaborative-filtering-in-29-lines-of-code/comment-page-1/#comment-75596 Chris K Tue, 28 Aug 2007 19:35:23 +0000 http://www.serpentine.com/blog/2007/08/27/weighted-slope-one-in-haskell-collaborative-filtering-in-29-lines-of-code/#comment-75596 And before I run for dinner I changed the code on the wiki so that update2 produces an un-normalized SlopeOne' matrix and changed my predict' to consume this instead. You are right -- the sparse matrices as nested maps are easy to work with. And I got to use Data.Map.mapMaybeWithKey and Data.Map.intersectionWith which I have not had call to employ before. Cheers, Chris And before I run for dinner I changed the code on the wiki so that update2 produces an un-normalized SlopeOne’ matrix and changed my predict’ to consume this instead.

You are right — the sparse matrices as nested maps are easy to work with. And I got to use Data.Map.mapMaybeWithKey and Data.Map.intersectionWith which I have not had call to employ before.

Cheers,
Chris

]]> By: Chris K http://www.serpentine.com/blog/2007/08/27/weighted-slope-one-in-haskell-collaborative-filtering-in-29-lines-of-code/comment-page-1/#comment-75585 Chris K Tue, 28 Aug 2007 19:02:35 +0000 http://www.serpentine.com/blog/2007/08/27/weighted-slope-one-in-haskell-collaborative-filtering-in-29-lines-of-code/#comment-75585 The blog ate my code, so I posted all my code at http://haskell.org/haskellwiki/CollaborativeFiltering The edited update function ensures there are no "diagonal elements". The predict' function makes the relationship of the SlopeOne matrix to the predicted value a bit clearer. The blog ate my code, so I posted all my code at http://haskell.org/haskellwiki/CollaborativeFiltering

The edited update function ensures there are no “diagonal elements”.

The predict’ function makes the relationship of the SlopeOne matrix to the predicted value a bit clearer.

]]> By: Chris K http://www.serpentine.com/blog/2007/08/27/weighted-slope-one-in-haskell-collaborative-filtering-in-29-lines-of-code/comment-page-1/#comment-75580 Chris K Tue, 28 Aug 2007 18:53:57 +0000 http://www.serpentine.com/blog/2007/08/27/weighted-slope-one-in-haskell-collaborative-filtering-in-29-lines-of-code/#comment-75580 Ah, I have now read your "predict" as well as the "update". I had to rewrite your variable names to be sure I could read your code correctly. Reading the original paper I see your algorithms look correct. The default (0,0) does nothing since the first 0 multiples the rating and so the running total is unchanged. A possibly clearer version of predict: predict' :: Ord a => SlopeOne a -> Rating a -> Rating a predict' (SlopeOne matrixIn) userRatings = M.mapMaybeWithKey calcItem matrixIn where calcItem item1 innerMap | M.member item1 userRatings = Nothing | M.null combined = Nothing | norm_rating Ah, I have now read your “predict” as well as the “update”. I had to rewrite your variable names to be sure I could read your code correctly. Reading the original paper I see your algorithms look correct. The default (0,0) does nothing since the first 0 multiples the rating and so the running total is unchanged.

A possibly clearer version of predict:

predict’ :: Ord a => SlopeOne a -> Rating a -> Rating a
predict’ (SlopeOne matrixIn) userRatings = M.mapMaybeWithKey calcItem matrixIn
where calcItem item1 innerMap | M.member item1 userRatings = Nothing
| M.null combined = Nothing
| norm_rating

]]> By: Bryan O'Sullivan http://www.serpentine.com/blog/2007/08/27/weighted-slope-one-in-haskell-collaborative-filtering-in-29-lines-of-code/comment-page-1/#comment-75557 Bryan O'Sullivan Tue, 28 Aug 2007 17:58:00 +0000 http://www.serpentine.com/blog/2007/08/27/weighted-slope-one-in-haskell-collaborative-filtering-in-29-lines-of-code/#comment-75557 Hi, Chris. Regarding your suggested stylistic change, I originally wrote the Haskell code to use the kind of map you suggest, but it was both longer and less readable than the current code, the reason being that the left-hand item in a key pair was overrepresented in proportion to the number of times the right-hand item showed up. Accounting for this was annoying. Using nested maps avoids the problem. Hi, Chris. Regarding your suggested stylistic change, I originally wrote the Haskell code to use the kind of map you suggest, but it was both longer and less readable than the current code, the reason being that the left-hand item in a key pair was overrepresented in proportion to the number of times the right-hand item showed up. Accounting for this was annoying. Using nested maps avoids the problem.

]]> By: Chris K http://www.serpentine.com/blog/2007/08/27/weighted-slope-one-in-haskell-collaborative-filtering-in-29-lines-of-code/comment-page-1/#comment-75536 Chris K Tue, 28 Aug 2007 16:52:47 +0000 http://www.serpentine.com/blog/2007/08/27/weighted-slope-one-in-haskell-collaborative-filtering-in-29-lines-of-code/#comment-75536 Because of the "norm" operation, your "update" function only makes sense if applied to "empty :: SlopeOne a" as its first parameter. So I would suggest that initial empty value by hard coded instead of a parameter to "update". And in this case "update" should be renamed to something more like "slopeOneFromRatings" If you actually want "updateSlopeOne" that adds novel user preferences to a non-empty slopeOne value then you need to reverse the "norm" operation before adding in novel information (and finish with the "norm" operation). Or you could never apply "norm" to the slopeOne information and always do the "norm" division when looking up values. A more stylistic change would be to make slopeOne into the type "M.Map (a,a) (Int,Double)" which takes the item pair as a single key. Your current code with nested maps could be made slightly more efficient by grouping the insertions so that each insertion did not always involve a lookup into both the outer and inner map (by grouping by the outer Map's key either per user (easy given the way you construct prod) or sorted across all the users). Also the (item1,item2) keys are irrelevant since the value is always zero. Because of the “norm” operation, your “update” function only makes sense if applied to “empty :: SlopeOne a” as its first parameter. So I would suggest that initial empty value by hard coded instead of a parameter to “update”. And in this case “update” should be renamed to something more like “slopeOneFromRatings”

If you actually want “updateSlopeOne” that adds novel user preferences to a non-empty slopeOne value then you need to reverse the “norm” operation before adding in novel information (and finish with the “norm” operation).

Or you could never apply “norm” to the slopeOne information and always do the “norm” division when looking up values.

A more stylistic change would be to make slopeOne into the type “M.Map (a,a) (Int,Double)” which takes the item pair as a single key. Your current code with nested maps could be made slightly more efficient by grouping the insertions so that each insertion did not always involve a lookup into both the outer and inner map (by grouping by the outer Map’s key either per user (easy given the way you construct prod) or sorted across all the users). Also the (item1,item2)
keys are irrelevant since the value is always zero.

]]>