New and Improved Charts – Now With More Listener Ratings!
Short version: We’ve added charts to the aptly named Charts page that include a list of the “best rated” podiobooks.
Longer version: Here was our dilemma — Book A has an overall rating of 4.9999 (out of five), where that average was figured from 100 ratings. Book B has an overall rating of 5.0, but that’s because only a single person rated the book. Those are extreme examples, but it’s tricky (neigh impossible) to come up with some fancy math that doesn’t skew the chart to one side or another.
So like any smart new company — we totally ignored the problem and went for simplicity. You can now see two new charts on our Charts page showcasing the top rated audiobooks on our site. The first orders them strictly by the overall rating, and the second orders them by the total number of ratings which contributed to the overall total. Confused? Look below. It’ll make all the sense in the world. And I blurred the results just to encourage you to visit the actual page itself. I’m sneaky like that.
To make even more sense, we decided to take a total transparency approach. Now in both charts, you see the overall raging and the number of times the book has been rated.
Of course, any ratings system is as good as the people who use it — and hopefully don’t abuse it. That last part may be wishful thinking, as there’s no way to completely stop gaming of the system. Right now, anyone subscribed to a book can rate it. But since we’ve made it pretty painless to subscribe to a book (all free of charge, in the event this is your first time here), it’s probably not that hard to game. So expect some additional “checks and balances” in the near future. We probably won’t make those filters transparent, as that sorta defeats the purpose of trying to catch the gamers. And it’s not a huge priority now. And with luck… we won’t need to make it one. So long as everyone plays nice and fair.
Questions or comments about the new system? Leave a comment for all to see. Wanna rat someone out for gaming the system. Might wanna report that through our feedback mechanism as to not ruffle feathers in the event that you are wrong.
February 22nd, 2007 at 1:12 pm
What about not listing a book until it has at least X people have ranked it (say 20)?
I think it’s a little strange to rate a book that’s in progress… Can you change your ratings once you finish the book? If I’m trying to use the ratings to find something good I can’t really trust them. What if something started great (and everyone ranked it as such) but turned out not great at all by the end.
February 22nd, 2007 at 1:35 pm
JP,
Both are valid concerns:
What about not listing a book until it has at least X people have ranked it (say 20)?
Any number you pick is arbitrary and delays books from getting listed. And the fact is that we just don’t have that many people rating books right now. Plus, this gives some great exposure to new books and back titles. If it’s worth of the rating, it’ll stay as more folks rate. If not…
I think it’s a little strange to rate a book that’s in progress… Can you change your ratings once you finish the book?
I don’t find it strange at all, and you can always change your opinion at any time and re-rate the book. It doesn’t double count. While I agree that you may not be able to accurately rate a book in a single chapter, the fact remains that many people do make up their minds about a book in the first few paragraphs. And I’m thinking of the inverse of your example. If we required everyone to listen to (or at least download) every chapter of a book before rating, how many negative reviews would we see? I’m not trying to capture negative reviews, but requiring someone to listen to the whole thing sure seems like you’re sample size of ratings would be positive, as those who bailed early for whatever reason can’t submit their opinion.
February 22nd, 2007 at 7:17 pm
Would it be possible to ask listeners to rate a book when they hit “cancel subscription”? The listener would be able to give a star rating for the book, enter a comment, or skip the rating altogether.
I think that would encourage more people to rate the books at the time that they decided they heard enough.
February 23rd, 2007 at 4:31 am
Hey Evo,
The number was arbitrary, but it’s a system I’ve seen work well elsewhere (sometimes with a too new to rank list).
I didn’t mean to suggest (and looking at it I’m sorry I worded it how I did) that you require people to finish something to rate it (especially since you can change your ranking). Maybe it would be possible to update the file telling you that you got to the end of a book reminding you to update your ranking. Anyway, I was just throwing it out there for discussion. I wouldn’t mind something noting which books are complete in the ranking list (maybe just an icon or something).
Anyway, thanks for adding charts!