While much has been written on the soft launch paradigm for mobile game releases, one item remains conspicuously absent from popular consensus: how much data is needed for a soft launch to be conclusive (and, by logical extension, how much a soft launch must cost).
Obviously the answer to this particular question will change on a case-by-case basis, but ultimately what is important to most developers is some sense of the minimum commitment (in terms of team focus, marketing budget, and foregone revenues from the game being globally available) necessary to make an informed decision on the viability of a game. That said, before considering the particulars of a soft launch, a developer should understand the overall goals of a soft launch. Ultimately, the purpose of a soft launch is to answer a few fundamental questions about a game:
1) How readily do people click on ads for this game? Is this game relatively easy to market?
2) At what scale does this game maintain profitable unit economics? How big can the game get, and how much money will it make at that scale?
3) With which demographic groups does this game best resonate? Do those groups represent a large total addressable market on a global scale?
4) How viral is this game?
(a fifth meta question might be: if the answers to any or all of the above questions indicate that the game isn’t ready for a global launch, do we believe it has the potential to ever be ready, or should the project be killed now?)
In answering these questions, a soft launch helps a developer determine the commercial potential of the game: the possible scale of the game, how much paid marketing will contribute to that scale, and what the demographics of that scale look like.
But notice that only question #2 is directly answered by in-game metrics (such as retention data, session frequency and length, etc.), and even then, those metrics are really only used to form an estimate of per-user lifetime revenue (versus acquisition costs) at scale, which is necessarily assumption-based (that is, it requires an assumptions-based model of how costs and monetization change as marketing campaigns get larger and broader). The other questions are all related exclusively to marketing and can more or less be answered directly in the soft launch.
So of the four fundamental questions that a soft launch should answer, only one of them requires in-game metrics to be answered, and even then, the answer also requires an assumptions-based model of how those metrics change with scale.
What this means is that a large, steady volume of traffic to vet game metrics over a long period of time isn’t really needed to faithfully and responsibly conduct a soft launch. What is needed is reasonably-sized cohorts of users per game update in a variance-minimization approach that simply delivers robust results that don’t change materially from update to update. This doesn’t necessarily have to be hundreds or thousands of new players per day over the course of a soft launch; it could be as few as hundreds cohorted weekly, assuming volumes are consistent per day and traffic quality is kept more or less stable. The goal is to be confident that the metrics gathered are reliable, reflect the quality of the game, and are not the result of some fluke (a local holiday, etc.). The goal of a soft launch is not to achieve such a high volume of traffic that gameplay metrics — which only inform one part of the global launch calculus — are utterly consistent on a daily basis.
Of course, for some studios, spending hundreds of thousands or even millions of dollars in marketing over the course of a protracted soft launch is immaterial relative to what might be spent in a global launch, so large, extended campaigns are justifiable (or simply not a consideration). And the level of completeness in which a game enters soft launch will have a huge impact on the amount of time it spends in that state: Zynga’s Dawn of Titans, for instance, has been in soft launch for more than a year, whereas Machine Zone’s Mobile Strike was in soft launch for four months Supercell’s Clash Royale was in soft launch for a mere two months.
Recent