Profiles in Popularity III--The Limits of Popularity
Nov 12 '00 (Updated Nov 16 '00)
Third in a Series
This has turned into a tag-team series of reviews by myself and fellow Computer Advisor and management consultant MobiProf. I refer you to the other reviews in this series, conveniently labeled Profiles In Popularity I by lap0530 and Profiles in Popularity II and IV by MobiProf (see the URLs at the end of this review).
By comparing notes, MobiProf and I discovered that both of us had reviews that by rights should have appeared on the Most Popular Opinions (MPO) page for Computers, but did not. It seems our reviews had become too popular. MobiProf has done a masterful job of exposing the flaws behind the MPO page for Computers. I salute his efforts and findings, and refer you to his review for details. This review extends the analysis to other MPO pages.
Popular Opinions?
Unlike the Most Popular Reviewer (MPR) pages, which list the most popular reviewers overall and by category, the MPO pages are only displayed by category. MobiProf’s analysis showed that Epinions’ MPO page for Computers actually did not show the most popular reviews, but rather those reviews whose page visits fell beneath a certain number, in this case probably 2,500. Surprisingly, his and my (and anyone else’s) reviews with more visits than that number disappeared from the MPO page! We verified this by examining our own page visits from our private accounts.
Longer-term Epinions members will remember that prior to the end of the bonus Eroyalty program and the beginning of Income Share, one could look not only at his or her own member and total visits for any review, but also at the same information for every other review. While it was painful and discouraging for a newcomer to see that someone else’s review had 250 member visits and 994 total, compared with my own measly 17 and 34, I was at least fairly certain of the accuracy of the numbers--since my (and I presume everyone else’s) totals were dynamically updated. This information motivated me to greater efforts, as well as pointed me to the reviews and reviewers with the highest totals to determine both the most popular topics and the style of writing that led to higher totals.
With the advent of Income Share and the revision of member profile pages into a public and private version, Epinions removed the ability of members to see each other’s member and total visits. Instead, a summary rating appears on the public profile, and only the individual member can access his or her member and total visit count from the private profile page. This has led to an increased reliance on the MPR and MPO pages for information as to which reviews and reviewers are most popular. But in the case of the MPO pages, this information unfortunately is badly flawed. After corresponding with MobiProf and reading his analysis of the Computer MPO page, I sought to determine the accuracy of the information on other MPO pages.
Other MPO Pages and Their Likely Limits
In MobiProf’s review, an analysis of the MPO page for Computers and e-mail correspondence with other members including myself revealed that reviews having more than 2,500 total visits did not appear on the MPO page. In a comment on MobiProf’s review, member kristinafh confirmed that she had experienced a “disappearing review” in a different category as well, and suggested that each category may have a different upper limit.
By examining each MPO page, I am able to offer the following likely limits of popularity. Some categories, such as Bags and Boating, have no data, and are omitted from this analysis. For each category, I show my guess as to the upper limit being used and the currently displayed visit total for the most popular review listed on the page (in parentheses).
• Community Center--750 (699)
• Autos--2,500 (2324)
• Beauty--200 (155)
• Books--550 (suggested by kristinafh) (522)
• Computers--2,500 (2471)
• Education--350 (344)
• Electronics--2500 (2212)
• Enterprise Hardware--100 (70)
• Enterprise Software--200 (148)
• Finance--700 (646)
• Food and Drink--300 (271)
• Games--600 (580)
• Health & Wellness 500 (413)
• Home & Garden--1200 (1179)
• Kids & Family--700 (689)
• Magazines--500 (465)
• Movies--800 (778)
• Music--600 (522)
• Musical Equipment--150 (120)
• Networking--200 (182)
• Office Equipment--200 (167)
• Outdoor--800 (703)
• Parks--100 (87)
• Pets--200 (168)
• Restaurants--300 (256)
• Services--500 (453)
• Sports--800 (738)
• TV Ads--300 (238)
• Travel--1500 (1456)
• Web services--250 (221)
MOST Popular or Not?
Further to test the theory of the missing popular reviews, I examined my own reviews in the Web Services category. As previously mentioned, when examining hit totals one is now restricted to his or her own reviews (or to information other reviewers are willing to share). This makes the investigation problematic, since one does not have access to server logs, though some cached MPO pages can be found and examined by use of a search engine such as Google.
I write reviews frequently in the Web Services category and wondered why some of my more popular reviews were not listed in the MPO page. Although I am listed as a Most Popular Reviewer in Web Services, my visit total is shown as a meager 676. When I visit the MPO page for Web Services, I see listed 37 reviews with visit totals between 167 and 221, and none of my reviews are shown as MPO. Yet when I examine my own private page, I see that of my 34 reviews in the category, eleven should appear as MPO. These reviews have visit totals ranging from a low of 196 to a high of 947. Furthermore, my hit total for the category is more like 5,000 visits rather than the 676 shown. Please understand: I am not complaining, just pointing out a serious defect in the MPO pages and verifying that the defect is not limited to the Computers category.
The Result of Misinformation
Epinions is using a cutoff number for each MPO category, and as kristinafh suggested, the cutoff varies by category. When selecting MPOs, Epinions selects the forty reviews with page visit totals less than or equal to that cutoff number. These reviews are listed on the MPO page, while reviews with more than that number of visits are “scrolled off” the listing. This would be I suppose the equivalent of retiring a jersey in basketball. The cutoff number for each category could either be hard-coded or could have been dynamically generated at one point but now is being treated as a constant. In popular categories such as Computers and Autos, the cutoff is higher, while in less popular or newer categories, the number is smaller.
Why is this happening? Probably, it is a simple programming mistake. I do not believe that Epinions would intentionally withhold information concerning the page visits of the most popular reviews. It is clear that the visit totals on the MPO pages are not updated dynamically. To prove this, visit and rate a listed review that you have not yet read or rated. Return to the MPO page and find that the member visits have not been incremented.
Epinions periodically updates the MPR and MPO pages with the totals at a given point in time, unfortunately using the cutoff number to eliminate more popular reviews, and these new “MPO” visit totals remain static until the next update. Interestingly, not all the information on the MPO pages is static. For example, I changed the product rating from five stars to four stars on one of my reviews appearing as one of the MPOs in Food and Drink, and the new rating appeared when the MPO page was refreshed.
Check Your Own Reviews--You May be More Popular Than You Think!
To list your own reviews by category to see if you have any missing MPO reviews because they have become too popular, you can type the following into your browser address window (making sure it fits on one line). You must be logged into your own account for this to work, and can only examine your own visit totals. If you copy or type the following verbatim into your browser address window, you will be redirected to my public profile page. Substitute your Epinions ID and the category label of interest for “webs,” which I use here only as an example:
http://www.epinions.com/user-lap0530?template=members/account_writing.html&nosum=1&topic=webs
I would be interested in receiving e-mail from any member who has reviews in any category with more than the number of total visits posted in the MPO pages and which therefore are not shown on these pages. This would serve two purposes. First, the estimation of the actual cutoff for MPO pages could be made more accurate. Second, I for one (and I imagine many other readers) would be interested in knowing which reviews are truly the most popular!
Still Some Problems
One of the remaining mysteries in this analysis is why certain reviews that have visit totals within the range shown on the MPO pages are still not listed. My own personal records indicate that several of my Computer and Web Services reviews, as well as one of my TV Ads reviews all are between the limits shown on the MPO pages, but do not appear on the pages. Therefore, I would also be very interested in receiving an e-mail message from anyone who has a review with visit totals between the posted limits on the MPO pages, but which does not appear on the MPO page.
The theory of a cutoff for each category explains why reviews with more than that number do not appear, but does not address the reason for unlisted reviews within the range. MobiProf had suggested that one possibility is that when selecting the MPOs, the Epinions’ computer would discard any other reviews with the same number of visits as another and would select the next review in descending order of page visits. Yet my examination of several of the MPO pages revealed that often reviews are listed with identical page visit totals. Since these page visits remain static until the MPO page is updated, these reviews must have had the same number of visits when the MPO page was generated, so the theory of discarding reviews with identical page visits is disproved.
Conclusion
This review and the previous review in this series have demonstrated that the MPO pages do not display the truly most popular reviews. In Part IV, MobiProf describes how the MPO pages are generated.
Links
Profiles in Popularity I--A Statistical Analysis of the Twenty Most Popular Epinions Reviewers--by lap0530
http://computerteacher.epinions.com/user-review-1D1F-2143A61A-39EDE1C5-prod4
Profiles in Popularity II--Why the Most Popular Opinions Do Not Appear on the Most Popular Opinions Pages--by MobiProf
http://mmmm.epinions.com/user-review-3572-2715339B-39FE1B32-prod3
Profiles in Popularity IV--How the Most Popular Opinions Pages are Created--by MobiProf
http://mmmm.epinions.com/user-review-43EB-1596A4E0-3A106D60-prod2
 |
|
|
|
|
|
|
Epinions.com ID:
lap0530
|
 - Top 1000 |
|
Location: Anderson, SC
Reviews written: 122
Trusted by: 332 members
About Me:
Psychology and management professor and business consultant
|
|
|
© 1999-2009 Shopping.com, Inc.