How We Got Our Sex Toy Scores

Our data set is made up of average review scores for a given sex toy across 200+ online stores.  But as you probably know, you can’t just do a simple average across all the stores. Since different stores have different numbers of reviews, a simple average cannot reflect the real average. It’s very similar to how GPA is calculated when different courses have different numbers of credits.

So we used a weighted average. The formula is: [(average review in store 1 * number of reviews in store 1) + (average review in store 2 * number of reviews in store 2) + … + (average review in store n* number of reviews in store n)]/(total number of reviews across all stores).

Of course, a review rating itself doesn’t accurately measure product popularity. So, we needed to do more than use weighted averages.  That’s why doing a factor analysis is a better solution.

We used two methods (principle factor model and maximum-likelihood factor model) in the analysis to cross-verify at the end. In both models, we used Average Review Score, Total number of Reviews, and Number of Stores that carry the product to estimate the underlying product popularity. The reason is that not only popular products tend to receive higher and more reviews, but also more stores sell them too. Other variables, like number of reviews per store only measures the popularity of the store rather than the products.

We asked the statistical modeling program we used to try to find three potential underlying factors (it’s a common practice to weed out inaccurate results) but both models only generated one factor. It is a very good result because it is very likely that the “Factor 1” (in red circle in the two charts below) is the product popularity we want!

In order to generate a popularity index, we used factor loadings (numbers in the green circle). Factor loading means how much each variable contributed to the underlying popularity index. The formula is: popularity = variable1 * its corresponding loading + variable 2 * corresponding loading + ….  I created two columns with this formula in the Google Spreadsheet (see below) after range of cost.

As you can see in the charts, the two factor analysis models generate two seemingly different results for loadings and the formulas in the Google Spreadsheet are different. That’s fine because they use different methods and it turns out to be great at the end. Since the formula is generated based on the products that have data, when we update data on products that don’t currently have data at large scale, then we may consider adjusting the formula. The existing formula, given its statistical significance (or validity), should stay the same even with a larger data set.

We created three columns at the end (see example on page 3). These are the rankings (from high to low) for each product based on our simple average review, factor analysis (principle factor), and factor analysis (maximum-likelihood). As you can see, the two factor analysis results rank the first 20 products in the exact order! This confirms the validity of our results. 

We used Stata, a professional statistical analysis software to conduct the factor analysis.