Soft launching a mobile title, which is the process of releasing a game in a test market and iterating upon it based on user data, has become an integral part of development strategy for large and small developers alike. The reasons are numerous:
- Ratings have a significant impact on an app’s discoverability, perception, and, ultimately, download volume, so a game needs to be tested by a sizable number of users before it is hard launched in order to make sure that people enjoy it. By soft launching in a test market, a developer can receive negative feedback – and address the sources of that feedback — without hurting its title’s long-term discoverability in the markets that hold the most revenue potential.
- User acquisition is expensive. By iterating upon a game in soft launch, a developer can ensure that the best possible product will be released in hard launch, increasing retention and therefore return on marketing spend.
- Until a large number of users has played a game, a developer has no idea how it will be received. Soft launches can reveal interesting gameplay trends or patterns of use that could not have predicted during development.
In order to take advantage of a soft launch, a developer needs to be able to capture and report on the data users will generate. A few months ago, I define what I called the Minimum Viable Metrics – the basic set of metrics needed to draw conclusions about the performance of a free-to-play game. By instrumenting the Minimum Viable Metrics in soft launch, a developer is empowered to base product decisions on a robust set of behavioral data points and not intuition alone.
The biggest hurdle some developers face in soft launch is composing a narrative from the metrics they’re tracking. The minimum viable metrics need to be taken as a whole to communicate the health and potential of a free-to-play game; looking at high-level revenue or retention metrics alone doesn’t provide enough context to evaluate the a title’s potential.
The metrics that matter in soft launch are those that communicate how well players understand the game’s core loop, how often and for how long they play the game, and how readily they tell other people about the game. The mosaic constructed by the minimum viable metrics helps developers understand these important patterns of use.
The minimum viable metrics are comprised of four metrics groups: Retention, Engagement, Virality, and Monetization. Each group consists of a set of metrics, and, taken together on a daily basis, the entire portfolio should provide a developer with a picture of where the game stands with users – and, perhaps more instructively, how it is improving.
In the retention group, the developer will want to make sure that users understand the game and find it appealing – this is communicated through early-stage retention metrics like Day 1, Day 3, and Day 7 retention. Additionally, the developer will want to track a tutorial funnel to identify weak points in the onboarding process. Depending on the length of the soft launch, the developer may be able to spot longevity shortcomings with longer-term retention metrics like Day 30 retention.
The engagement metrics should track average and median sessions per day and session lengths. The difference between the average and median values is important; a large gap speaks to a minority of highly-engaged users playing the game for long stretches at a time, which could run counter to how highly-engaged users are meant to interact with the game. Additionally, the engagement metrics should be used to inform the balance the developer hopes to achieve between session frequency and session lengths.
In the virality group, the developer should be tracking how often any viral invitation mechanics are utilized by users, which is a good proxy for delight – if a user is willing to vouch for a game by inviting a friend to it, it’s a good bet that the user finds the game fun. The developer should also track a rough k-factor metric by evaluating non-paid user base growth as a function of virality. K-factor is an important concept to understand when the game is ready for hard launch because it can be used to estimate total return on a launch marketing budget.
The monetization metrics group should track high-level revenue benchmarks such as average revenue per daily active user (ARPDAU), average revenue per user (ARPU), and average revenue per paying user (ARPPU) as well as the distributions of the purchasable catalogue: first purchases by users, most common purchases, and the proportion of total revenue that each item in the catalogue represents.
By focusing closely on a baseline set of metrics in soft launch, a game developer can sidestep many potential land mines faced in hard launch with products that haven’t been thoroughly vetted by users. Free-to-play is heavily dependent on data-driven decision making and quick product iterations: by guiding development with the minimum viable metrics in the soft launch, a developer illuminates many of the unknowns that plague games released to the global market before they have been subjected to the crucible of App Store scrutiny.
