When browsing social media and adjacent sites, I’ve always wondered how ratings are implemented. This may sound pretty boring, but let me raise an important example: Imagine you’re browsing a community and searching for the most popular posts. In most cases, you’ll only find recent posts because the community has grown over time.
Let’s take a step back for a second.
Usually, social networks add some sort of rating mechanism, be it likes, upvotes, etc. to allow users to rate user-generated content. And with that often comes the feature to sort content by popularity. But how do we define popularity?
👍 Like
We could just order by likes (I’ll use that term throughout the rest of this post, feel free to swap it with your preferred unit), but communities rarely hover around a constant size, they expand and contract over time. This causes like inflation, where older posts are automatically worse off if a community has grown.
Let me illustrate this effect with a short example:
Imagine we have a community of cycling athletes who post their tours and stats. At the moment, 50 people participate. I just completed a huge tour and post a summary, which 30 people like. After a week, this post is in the top 10.
After a month, the community starts to grow, as someone shared the link on another social network. The community reaches 500 people. On average, posts receive 100 likes.
If we assume the larger community to be similarly active in posting content and I were to sort by most popular, I wouldn’t be able to find my post anymore.
Now, there are two sides we can take here: Maybe it’s good that larger communities can overrule previous trends with their vote share, but what if we lose timeless posts that would be equally valuable?
There are certain methods we could use to adjust for this effect. Of course, this depends on how you want to define popularity or relevance. Maybe it’s alright for you to rely on absolute metrics, maybe you want to be more nuanced.
📣 Participation over Likes
We can control for the growth in community size by measuring not only the absolute amount of likes a post received but also the community size at that point in time. If we combine those two metrics, we receive a measure of participation: Which share of the community liked this post?
By applying this transformation, we’ve gone from an absolute metric to a relative measure, a ratio. This can be compared over time, regardless of community size. It’s effectively unitless or normalized.
Posts often gain likes over time, sometimes posts are even discovered and gain traction years after their original publication. In this case, we could snapshot the highest engagement a post got (at the most viral point, this post was liked by % of the community). If a post only got upvoted by 10% of the community on the day of posting and subsequently reached 80% a month later due to some external event, we have to account for that.
In comparison, likes are easy to manage over time because they do not require re-calculations.
⏱ Virality or time-dependent engagement
Another dimension we can look at is time. Once a community establishes a trend or a post goes viral, engagement can be highest in the first hours and days. Sometimes, content is stale after a couple of days. Popular content could be on either extreme end: Some posts go viral and are forgotten in a week, and other posts may slowly increase in popularity. Both could reach the same level of likes, so we wouldn’t distinguish, but maybe your platform prefers one of the extremes?
To measure popularity by acceleration and growth rate, we need to include time. This could be in the form of milestones (how long until the post reached x% of participation), but also growth (x likes per hour).
🍲 Weighted metrics
As a first step, we can take different metrics into account. Maybe there’s more than just likes, comments, and engagement (watch time, completion rate, retweets, etc.) that could all be factored in when it comes to popularity.
If we have multiple metrics to include in our popularity ranking, we could easily use the mean or a weighted average, if we require different weights. For that, all values should be unitless (i.e. use the same scale). One possible method is to normalize the number, by which we obtain a value between 0 and 1.
I think we can agree on the fact that there are infinite ways to measure popularity and as so often is the case, there’s no one-size-fits-all approach. Pick what supports your values and make sure to keep refining your models (especially when you want to prioritize long-term value creation over short-term virality metrics).
Also, note how I haven’t talked about ML so far. Surely, there are ways to bring that in, but I think, for platforms just getting of the ground, finding a good measure mostly requires explorative and descriptive statistics, not models where we can no longer connect inputs and outputs. (I’d love to be proven wrong about this, of course, and I haven’t forgotten the algorithmic recommendation system that powers the infinity pool of lost hours over at TikTok).