Scoring Algorithms
I’ve had a few projects where I wanted to come up with a scoring algorithm. I ususally head down the path or try to multiply averages or up-down ratings some numbers with some weights, and sometimes it works, but I want something that feels better.
Here’s a Google Sheet I made to play with a few different scoring systems.
https://docs.google.com/spreadsheets/d/1SFCv5XKfeA41g49DUYZuanoNjPnNzoqEMCsyaaFDpcA/edit?usp=sharing
Wilson Confidence
Popular from Evan Miller’s posts, but a bit complicated.
Laplace Smoothing
This basically makes an assumption that all the things you’re ranking start with some number of upvotes and downvotes. Greatly simplifies the math required to get a good scoring system going.
$$rating=\frac{(up + default)}{((up + default) + (down + default))}$$
You can just plug in a constant for default.
So for something with no votes, and a default of 7:
$$rating=\frac{(0 + default)}{((0 + default) + (0 + default))}$$
$$rating=\frac{default}{(default + default)}$$
$$rating=\frac{7}{14}$$
$$rating=0.5$$
Different default values for up and down votes
Similar to the Smoothing used above, but uses different default values for up and down votes.
$$rating=\frac{(up + upAvg)}{((up + upAvg) + (down + downAvg))}$$
As an example for something with 13 up votes and 2 down votes. We’ll assume everything by default has 4 upvotes and 7 downvotes.
$$\text{score}=\frac{(up + 4)}{((up + 4) + (down + 7))}$$
$$\text{score}=\frac{(13 + 4)}{((13 + 4) + (2 + 7))}$$
$$\text{score}=\frac{17}{26} = 0.6538$$
See my Cocktail App - Recommendation section for a real example of using this for a scoring application.
Resources
- https://www.evanmiller.org/how-not-to-sort-by-average-rating.html
- https://www.evanmiller.org/bayesian-average-ratings.html - followup to the previous post.
- https://julesjacobs.com/2015/08/17/bayesian-scoring-of-ratings.html - a follow up to Evan’s post, Julie also provides some simplifications to the formulas given by Evan. I ended up using this for the cocktail app.
- https://www.codementor.io/@arpitbhayani/solving-an-age-old-problem-using-bayesian-average-15fy4ww08p - Solving an age-old problem using Bayesian Average - Good writeup starting with simple averages, and getting more complex with bayes averaging.
- https://fulmicoton.com/posts/bayesian_rating/ - Math formula heavy here, but also breaks things down really well.
- https://planspace.org/2014/08/17/how-to-sort-by-average-rating/ -Great post on laplace smoothing
- Bayesian Ranking System Ranking with varying numbers of responses
- How to Count Thumb-Ups and Thumb-Downs: User-Rating based Ranking of Items from an Axiomatic Perspective - Whitepaper from 2011.
