What is Creative Optimization for User Acquisition?

Test. Optimize. Deploy. Repeat.

This is the virtuous mantra that digital and growth marketers stand by. But how does one implement it? What are the steps? What tools are available to conduct these experiments? How do we measure the uplift in performance?

Before we get started, let’s delve into why A/B testing is necessary and how it helps in maximizing campaign performance.

Creative Optimization Framework

In every collection of creatives there will be a few that perform well and others that do not. Ensuring that users are shown the better performing creatives will better attract the intended audience for your app (and increase conversions).

For this to happen, new creatives need to be added regularly while creatives that perform below par need to be removed.

Note: A creative variation (variant) contains variations to one or more of the following elements:

• Headline
• Subheadline
• Paragraph text
• Call to Action text
• Call to Action button (size and color)
• Images
• Logos
• Inclusion or exclusion of App Store / Google Play store badges)
• Background

CTA link (landing page / deep link) is, strictly speaking, not part of the creative elements that goes into a variant, and hence not considered part of the creative optimization process.

Defining the Optimization Goal

It’s important to determine the parameters the creatives will be optimized or ranked on.

Rather than go with Click-Through Rate (CTR), or Installs per 1,000 impressions (IPM), or even Click to Install (CTI), we recommend optimizing on Actions per 1,000 impressions (APM).

For example, Lifetime Value per 1,000 impressions (LTVM) is an APM metric. This effectively means that under performing creatives are removed based on LTVM metric performance.

This is because APM goals are closer to and reflect your true business objectives. For example, if you take a CTR uplift goal, it is possible the optimization efforts takes us towards more clickbait variants that are not high “value.”

This post will describe a means to optimize creatives on the basis of APM uplift.

Brute Force Approach to Creative Optimization

The brute force approach (Buffalo theory) is extremely effective for CTR-based creative optimization. However, it is infeasible when it comes to optimization on an APM metric.

This is because the time period involved for, say, LTVM optimization is usually up to 14 days,* which makes the optimization process extremely expensive and time consuming.

* Considering users download an app up to seven days after an ad impression - the standard attribution window - and allowing another seven days for users to perform the intended post-install actions (or LTV events) to get recorded.

However, if you are indeed optimizing for CTR (for example, drive traffic campaigns) and the number of creatives to be tested is manageable, then this approach is incredibly effective.

It involves launching a campaign with all creatives simultaneously. Once all creatives get a certain minimum and largely equal exposure, the CTR is calculated for each creative.

Post that, underperforming creatives are identified and paused. For example, one could pause all creatives whose CTR is below a certain absolute threshold or a certain ratio below the best performing creative’s CTR (say, 80 percent).

The aim is to retain a certain minimum number of creatives to ensure creative diversity, and then create more variants of the best performing creative and add it to the campaign to further optimize on elements within the creative theme.

Having said that, if you are using dynamic creative optimization (DCO) tools, which effectively means having a large number of creatives to test, this approach is sub-optimal.

Note: A key assumption here in the setup described is that all creatives are intended to target the same audience and the delivery optimization event selected is the same.

Introducing the Buffalo ‘Blitz’ Optimization Technique

The above mentioned brute force technique can be adapted and tweaked to make it work for APM uplift goals.

The constraints on time, resource and budget can be overcome with a ‘divide and conquer’ approach that we refer to as the Buffalo Blitz.

Setup and Process

Requirement: At least four creatives are available.

1. Select at random any four creatives.
2. Launch a campaign with the selected creatives. Ensure each creative gets a certain minimum (and largely equal) exposure within a defined period to reach statistical significance. Note: Exposure can be defined in terms of impressions, clicks, installs, actions or a combination of these. For example, consider a rule such as this: 10,000 impressions or 500 clicks or 25 installs or five actions - whichever is earliest.
3. Calculate the LTVM metric for each creative (take a rolling window of 14 days).
4. Pause all other creatives except the best performing one (based on the LTVM metric). Essentially, retain the top performing creative and pause the rest.
5. Add three new creatives selected at random to the campaign.
6. Repeat step three.

The process never ends - keep testing. As each new variant goes live, test elements of your creative which contribute to maximum impact.

Repeatedly doing this process helps make sure that only the most engaging creative variant(s) are being served.

Advantages of Buffalo Blitz

This tiered approach helps to quickly determine a “winner” and iterate efficiently.

At the same time, it also mitigates any negative impact to upstream metrics such as sudden drops in conversion rate. It does this by efficiently allocating budgets for maximum APM uplift with minimal campaign changes (effort). This results in increased ROI.

Further Optimizations on Buffalo Blitz

The ‘divide-and-conquer’ approach can be further efficiently implemented by a small tweak to the creative selection process.

First, group the creatives into themes: a distinct group of creatives based on a concept that is markedly different in messaging and visual from other themes. Each theme will only contain a set of creative variations under them.

Then, instead of selecting any four creatives, select at random four creatives in total across themes, ensuring at least one creative from each theme is included.

The reasoning behind this is based on the intuition that variance in the APM metric will be higher across themes than across creatives within a theme.

Also, when randomly selecting the next three creatives, preference is given to creatives from new themes. If none exist, then add the variants of the best performing theme.

It’s possible for variants of other themes to go untested which could have resulted in better performance. But this can easily be solved for by including those creatives where the variance between themes are comparable.

Things to be Wary Of

Certain types of creative themes such as those with offers, discounts or rewards tend to do better than creatives which do not carry any incentives.

Also, do not reject any creative ideas before testing. While the motivation is certainly to reduce costs, uncertainty and complexity, one needs to guard against such biases.

Creative Optimization for User Acquisition (UA) is a continuous process of incorporating the top performing creatives in campaigns by testing and iterating Creatives & Ad Formats for achieving the UA objectives.

The decision to 'Click' on an ad and then 'Install' the app is highly influenced by UA Creatives. Even the best UA strategies backed by strong technology require optimized UA Creatives to yield the desired results.

Two important KPIs to measure for Creatives Performance are:

1-Click-Through Rate (CTR)

It measures what percentage of potential users clicked on the ad after seeing ad impressions.

CTR = (Total Clicks / Total Ad Impressions) x 100

It reflects how attractive the ad is to the target audience and whether it generates any interest in them to know more.

2-Conversion Rate (CVR)

It measures what percentage of Clicks lead to the desired ad objective (often App Install for mobile apps).

CVR = (Total Installs / Total Clicks) x 100

It reflects how effective the combination of UA Creative and App Store listing is towards influencing Installs.

Below is an example for how Creatives affect Cost Per Install (Creative B is the winner in this case):

How to optimize UA Creatives?

1-Test multiple formats

With multiple ad networks, platforms, geos, and target audiences, depending on a single ad format is not advisable. Consider testing different ad formats like Static Images, Video Ads, Banners, Native Ads, Playable Ads, and more.

Ad networks keep introducing new ad formats periodically. You can consider testing new formats when they are launched. The competition will be low during the initial stage. You can experiment & learn at low scales to be ahead of the competition.

2-Test multiple variants of creatives

Within multiple formats, test creatives based on different creative elements, art style, messaging, text, duration(for videos), etc.

3 - Localization

Localizing creatives doesn’t just mean localizing the text. We have to culturize it too. Different countries react differently to ads and we need to adapt our creatives strategy accordingly.

Some of the benefits of Creative Optimization:

• Achieving lower Cost Per Install (CPI)

• Scaling the UA at relatively lower CPI levels

• High engagement with the ads helps in getting the most number of early impressions as compared to the competition

• The quality of installs will be high (better retention/engagement/monetization)

Some points:

• Higher CTR may not always lead to higher CVR. If CVR is low, then either the UA creative is not communicating the essence of the game or app store page elements have to be improved.

• Misleading ads with attractive elements may help in getting the installs at low CPIs but they later result in bad ratings & reviews or lower retention.

• Top performing creatives may hit saturation over a period of time due to Creative Fatigue. Test and iterate continuously.

• Stay ahead of the completion through innovation in UA Creatives concepts. Top performing creatives are often copied by the competition sooner or later.

• Use the creatives relevant to your Target Audience.