The pitfalls of industry benchmarks


Mobile analytics firm Swrve released an interesting report last week which presented a set of provocative statistics about games in its network:

  • 19% of players opened the game only once;
  • 66% of players didn’t return to the game after 24 hours from install;
  • 53% of total spending occurred in the first week of gameplay.

(Swrve regularly releases similar reports and whitepapers; its archive is worth reading in its entirety)

The report was interpreted in the press and on Twitter as a gloomy if not hopeless bill of health for the mobile gaming industry: massive bounce rates, overwhelming churn after day one, and the majority of all spending taking place in just one week.

Fortunately, these metrics don’t proxy anything about the mobile gaming industry as a whole; no metrics can. Each game (and, more broadly, app) in the App Store has a different profitability profile, and the metrics that comprise that profile (DAU, retention, average monetization) can’t be evaluated against a universal mobile gaming standard.

Broad, industry-wide performance metrics are meaningless because different games appeal to differently-sized demographics, some of which are cheaper to market to than others. Mobile app metrics are moving targets; taking a snapshot of a broad portfolio doesn’t reveal anything about those games individually, nor anything about the industry as a whole. Ultimately, an app’s metrics are primarily a function of the aggressiveness and focus through which it targets users.

In other words, a mobile app doesn’t possess some intrinsic LTV or retention curve; those metrics are defined by app developer’s ability to recruit appropriate users. It’s the marketing, stupid.

Marketing – or, more specifically, a developer’s ability to target and market to users that will enjoy its app – essentially defines an app’s retention curve and LTV. Broadly-targeted marketing campaigns meant to reach large swaths of mobile device owners generally go hand-in-hand with low early-stage retention, as people that don’t necessarily fit the profile of the app’s core user demographic are targeted to. This is the click-through rate conundrum: increasing a marketing campaign’s click-through rate (appeal) doesn’t necessarily result in lower per-install marketing costs.

But such campaigns can still be profitable, for a number of reasons:

  • High virality reduces eCPI to a profitable level, despite the dropoff;
  • High monetization of the users that do stay justifies the “wasted” marketing spend;
  • Chart visibility offsets marketing spend with organic installs (as in a “trampoline launch”).

Because of these factors, retention rates – out of context – don’t really speak to the profitability or even appeal of an app. The more targeted a marketing campaign is, the higher that cohort of users’ retention should be: consider a marketing strategy whereupon a developer walks the streets showing people how to play a game and subsequently installs it on their phone if they really like it. While Day One retention for that cohort of users would likely be close to 100%, it’s unlikely the campaign would achieve a meaningful profit.

While such a campaign might seem like hyperbole, some campaigns approach that level of interaction in driving installs; for some apps, that degree of expenditure is justified by monetization. For other apps, a high level of targeting isn’t needed because the total addressable market for the app is so large and varied. And for every app in between those two points on the spectrum, there’s an optimal level of targeting and reach that delivers an acceptable amount of profit. That optimal level may produce Day One retention of less than 40%.

In late 2013, market research firm Superdata released a report which portended an inevitable “bloodbath” in mobile user acquisition. The report indicated that average CPIs had exceeded average LTVs for the second half of the year, which the company interpreted to mean that mobile user acquisition was being conducted unprofitably industry-wide and must be heading for an implosion.

Of course, the implosion never came, because those industry-average metrics were contextless and therefore uninformative. Some developers can run profitable, in aggregate, campaigns in which CPI exceeds LTV. And some developers don’t have to; there is practically no cap to what they can pay for advertising placements because their apps monetize so well. The reason the numbers don’t scale up to the industry level is that the best-monetizing apps run massive user acquisition campaigns that reach millions upon millions of users per quarter: in 2013, King, the developer behind Candy Crush Saga, spent nearly $400MM on marketing according to its F-1 filing.

But how could profit be generated from a marketplace in which average install prices exceed average lifetime customer values? The answer, as before, is virality and chart dynamics, as well as the misleading nature of averages in an industry where very little of anything is normally distributed and where large developers benefit from information and distribution advantages.

Ultimately, the only metrics relevant to an app developer are its own, within the context of its broader marketing and content development strategy. Broad industry benchmarks aren’t helpful.