Our ranking system is pretty simple and is mostly based on reviews from users and experts. We’ll try to explain how it works in more detail.
So the two major components of a score are user reviews and expert reviews. User reviews are the scores you usually see (and leave) on online stores like Amazon. These are regular users like yourself who bought a toy and decided to share the experience with their fellow buyers.
We combine the user scores from all available stores we could find and calculate the average user score for each product.
The weight of user reviews in the final product score is 65%.
On contrary, expert reviews are written by, well, experts. Or at least by those we consider experts. We collect all expert reviews we could find (and think are credible) and calculate the average expert score. We also apply the weighted coefficient to the score of each expert based on expert’s trust, experience and popularity. This just means that the review from a blogger with 100k+ subscribers on YouTube, who made 200+ reviews and has been blogging for 10 years, influences the final product score more than the review from a blogger that wrote 10 reviews and started only a year ago.
Also, if the expert doesn’t provide any numerical rating for the product reviewed, we do it ourselves based on the content written (or filmed) by the expert. If you’re among the experts whose review we used and if you don’t agree with the score we assigned to the product based on your review, just let us know and we’ll make the necessary adjustments.
The weight of expert reviews in the final product score is 35%.
To cut the story short, Average user score * 65% + Average Expert score * 35%.
So the formula is extremely simple, but we may make additional adjustments to the score.
First, we adjust the score depending on the number of user reviews:
- Number of user reviews < 10 – final score minus 10
- 10 < Number of user reviews < 50 – final score minus 5
- 50 < Number of user reviews < 500 – no adjustment
- 500 < Number of user reviews – final score plus 3
Yeah, we punish the low number of reviews more than we reward a high number of reviews.
Next, if the product doesn’t have any expert reviews, we subtract 5-20 points from the final score depending on the number of user reviews. Why 5-20 you ask? Imagine we have a product with no expert reviews and one user review score of 5.0. According to the main formula, the product would receive a score of 90 with just one review, subtract another 5 and we receive a score of 85 which is still very good for the product with just one review, so in such cases we may decide to adjust the score even further. There aren’t many products without expert reviews so you shouldn’t worry too much about us adjusting the score too much.
Also, sometimes the expert reviews we use are just partially “expert” or “reviews” in case they are done by stores themselves or mostly examine the outer characteristics of the toy. We assign low weight to such reviews so they don’t influence the final score too much. Don’t worry about that.
One more adjustment we may make is based on the source of user reviews. If the majority of user reviews come from the toy brand website, we may lower the weight of such reviews and adjust the score. Why do we do so? Because the toy brands have control over the reviews that are left on their websites. Sure, the brands present on our website would never do such treachery but you can consider this as our little whim.
Next, users are welcome to leave toy reviews and questions on our website as well. Moreover, you are more than welcome to do so. We add these reviews to the evaluation process as well.
Finally, as toys users ourselves, we may adjust the final score based on our personal experience with the toy.
Hopefully, this clears it all. Basically, it’s just one simple formula and a bunch of adjustments described above. If you have any questions or suggestions, let us know. The ranking methodology is subject to change as we’re constantly improving it.