http://www.programmersheaven.com/ Contains the latest posts from the thread 'pearson algorithm for finding similarity' posted on the 'Python' forum at Programmer's Heaven. en Copyright 2012 Programmers Heaven Wed, 23 May 2012 23:59:23 -0700 Wed, 23 May 2012 23:59:23 -0700 Argotic Syndication Framework 2007.3.0.1, http://www.codeplex.com/Argotic http://www.rssboard.org/rss-specification 360 http://www.programmersheaven.com/images/ph.gif http://www.programmersheaven.com/ 88 31 http://www.programmersheaven.com/mb/python/414413/414413/pearson-algorithm-for-finding-similarity/ we hav used pearson algorithm which is given below. Ideally the algorithm should return values between -1 and 1 but since we hav a large set of data therefore it is giving values such as 1.0327955589886444 and 1.1547005383792517 so can u suggest us some solution to this problem or any other efficient algorithm for finding similarity between users.<br /> <br /> <br /> the input file is a dictionary of users, their choices and ranking.<br /> eg: dict1={user1:{choice1:rank1,choice2:rank1},user2:{ choice1:rank1,choice2:rank1}, user3:{choice1:rank1,choice2:rank1}}<br /> but we are working with a very large dictionary consisting of varying ranking.<br /> <br /> def pearson(prefs,p1,p2):<br /> # Get the list of mutually rated items<br /> si={}<br /> for item in prefs[p1]: <br /> if item in prefs[p2]: si[item]=1<br /> # if they are no ratings in common, return 0<br /> if len(si)==0: return <br /> # Sum calculations<br /> n=len(si)<br /> # Sums of all the preferences<br /> sum1=sum([prefs[p1][it] for it in si])<br /> sum2=sum([prefs[p2][it] for it in si])<br /> # Sums of the squares<br /> sum1Sq=sum([pow(prefs[p1][it],2) for it in si])<br /> sum2Sq=sum([pow(prefs[p2][it],2) for it in si]) <br /> # Sum of the products<br /> pSum=sum([prefs[p1][it]*prefs[p2][it] for it in si])<br /> # Calculate r (Pearson score)<br /> num=pSum-(sum1*sum2/n)<br /> den=math.sqrt((sum1Sq-pow(sum1,2)/n)*(sum2Sq-pow(sum2,2)/n))<br /> if den==0: return 0<br /> r=num/den<br /> return r<br /> http://www.programmersheaven.com/mb/python/414413/414413/pearson-algorithm-for-finding-similarity/ Wed, 10 Mar 2010 21:34:25 -0700 Python http://www.programmersheaven.com/mb/python/414413/415068/re-pearson-algorithm-for-finding-similarity/#415068 The correlation coefficient is really just a cosine of the angle between the two arrays. Here is a link to some code for computing it:<br /> <br /> <a href="http://www.dreamincode.net/code/snippet3042.htm">http://www.dreamincode.net/code/snippet3042.htm</a><br /> <br /> If your values are outside of the range of -1 to +1, then either you are computing it incorrectly or you have some pretty profound round-off error.<br /> <br /> You can see more about the formula in Wikipedia:<br /> <a href="http://en.wikipedia.org/wiki/Correlation_and_dependence">http://en.wikipedia.org/wiki/Correlation_and_dependence</a><br /> <br /> and more about cosine and correlation here:<br /> <a href="http://www.mega.nu/ampp/rummel/uc.htm">http://www.mega.nu/ampp/rummel/uc.htm</a><br /> Herb<br /> http://www.programmersheaven.com/mb/python/414413/415068/re-pearson-algorithm-for-finding-similarity/#415068 Wed, 31 Mar 2010 08:26:28 -0700 Python