Producing and deploying advertising creative at scale

The broad transition within mobile advertising into automated campaign management and optimization catalyzed a tectonic shift that has fundamentally transformed the way mobile-first companies build marketing teams and pursue product growth. I have written about the broader implications of the mobile app economy’s second act — to which automated marketing campaign management certainly has contributed — but one aspect of this change that is usually only discussed in the abstract is the outsized role that ad creative asset production now plays in mobile marketing.

I have seen advertisers struggle with ad creative production in this new algorithmic marketing paradigm — in my work with my consulting practice, Heracles, as well as in the Mobile Dev Memo slack team and on QuantMar — but very little valuable insight exists publicly on how to manage an ad creative process for modern mobile marketing.

The importance of creative has increased not merely because campaign management has been automated away and thus there’s nothing else for a marketing team to focus on. The importance of creative production has been amplified because automated campaign management unlocks so much more value from ad creative than in the past. In this post, I’ll explain why ad creative plays such a large role in mobile marketing success and provide a framework for operating an ad creative production and testing process at scale.

Why is creative so important with automated campaign management?

A first thing to clarify in starting the post is that ad creative is not suddenly important because Facebook and Google automate campaign management: ad creative production has always been critically important to mobile marketing teams. What has changed now is that Google and Facebook can target and segment groups of users so much more specifically and efficiently with their automated optimization schemes — AEO and VO on Facebook and Google’s UAC — that testing very many ad creative variants against every segment of potential audience is now possible.

In other words: in the manual campaign optimization setting, a team would need to create some unmanageable number of user segment targets in order to test specific creative against all of them, which is impractical: it’s time consuming to create each ad set, and targeting parameters are generally set ex nihilo, meaning a media buyer would need to think up combinations of interest and demographic targets to potentially use with an ad set without much objective insight into how they might perform.

When a marketing team was tasked with creating targeting segments manually, they didn’t test every possible combination because doing so would be 1) very cumbersome and time-consuming, 2) expensive (it costs money to test!), and 3) likely unprofitable (the vast majority of segments won’t perform well). Now, Facebook and Google construct these segments for advertisers based on insight into how other apps perform but also based on very rapid evaluation of performance benchmarks (clicks, video views, etc.) that gradually progress towards ROAS (see this QuantMar thread for more background on how Facebook’s targeting algorithm works).

The point here is that, while creative was always an important input to marketing success on mobile, with manual segment targeting, it wasn’t feasible to test creative against every possible user segment — but now it is. And because of that, there is a direct link between the volume and diversity of creatives and the success of a campaign: the more varied creatives an advertiser is able to produce, the more granularly and specifically the various segments that Facebook and Google construct are able to be evaluated, leading to optimal performance at the level of very targeted segments versus sub-optimal performance at the level of the very broad segment definitions that advertisers would construct manually.

As an example, consider these three scenarios: an optimal creative-segment pairing configuration in the old, manual creation paradigm; a sub-optimal creative-segment pairing configuration in the new, automated paradigm; and an optimal creative-segment pairing configuration in the new, automated paradigm:

There are three key takeaways from this new paradigm that advertisers need to internalize:

1. The ad platforms are creating many more targeted user segments than most advertisers had previously advertised to;
2. Advertisers are blind to the segment construction, meaning they have no idea how the segments are defined (ad platforms aren’t transparent with the dimensions they use to define segments beyond the definitions that advertisers supply);
3. The only way to optimally advertise is to test very many ad creatives so that ad platforms can pair the most resonant creatives with the most appropriate segments.

I refer to the fact that advertisers are blind to the way segments are constructed by ad platforms as the Audience Targeting Iceberg: an advertiser obviously knows the high-level demographic and interest targets that it supplies to ad platforms for broad targeting, but they are blind to all of the features that ad platforms use to segment audiences within those constraints. The features that the advertiser can’t see far outweigh the ones it defines:

It’s important to call out the fact that these features are blind to the advertiser because the crux of this automated pairing mechanic is that an advertiser can’t pre-empt the segment definitions with intuition like they did with manual segment creation — an advertiser has no idea how the segments are defined, and so they can’t use intuition to build appropriate ad creatives for them a priori creative performance metrics. The only way for an advertiser to optimally pair creative with the segments that ad platforms algorithmically define is to test a massive number of ad creatives. This is why rapid creative production, iteration, and experimentation is such an important part of modern mobile marketing.

Testing ad concepts and variants

There is an important distinction to make here between creative concepts and creative variants. I define a creative concept as the narrative structure of an ad: it is the “plot” of the creative, or the story that it is telling about the product. I define an ad variant as some arrangement of a concept that is visually, perceptibly different from other arrangements.

The challenge in building a scaled, efficient creative production process is often more related to consistently coming up with new concepts than it is of delivering variants on concepts. Generating ideas for creative concepts is difficult: manifesting a whole, unique idea from nothing is arduous. Some advertisers look to popular creative themes for inspiration (see this QuantMar thread about user acquisition themes for some examples), and that can be a fruitful strategy, but the real goal in producing a consistent stream of new ad concepts should be to come up with an idea that outperforms not just an advertiser’s existing portfolio of ad creatives but of everything in the market — the advertiser should seek to introduce a creative that stands out among competitive ads and creates outsized results for the advertiser.

In this QuantMar thread, I introduced a simple framework for testing ad creatives: starting with concepts, progressing into themes, and ending with variants:

This framework allows the advertiser to be sure that the best possible version of an idea is surfaced from amongst many possible permutations of it. This is no easy task: each of the steps in this process involves not only art production but also ad spend and analysis.

Looking at the iterative, funneled nature of creative testing should bestow some appreciation of the enormity of the task of consistently delivering performant ads: it’s slow, time consuming, and expensive. But a streamlined and efficient creative production and testing process is also a massive competitive advantage, because building a conveyor belt of outperforming ad creatives is the only way to maintain an elevated average level of advertising performance.

What’s the point of new creative?

Digital advertising is a mean-reverting system. Within an advertiser’s portfolio, new ads tend to outperform old ads simply because they are new: users are more receptive to something they are seeing the first time than to something they have seen multiple times by definition. This dynamic is captured in the notion of ad saturation: after enough time, it’s possible that the only people who might see an ad are those that have already seen it.

What an advertiser should seek to do with an ad creative production process is to increase the mean performance of its portfolio of ads by replacing ads before they start to degrade. In this QuantMar thread, I talk about the ad creative lifecycle and what I call a creative’s “Half Life”: the point after which performance degrades precipitously because of saturation.

By pre-empting a creative’s half life with another creative — that is, by replacing an active, deteriorating creative with a new creative before its performance degrades dramatically — an advertiser can increase the mean creative performance across its portfolio of ads. The important aspect of this notion is that creatives should be replaced before their performance collapses — in order to do that, the production of an ad creative’s replacement should be started as soon as that creative is launched. To illustrate this concept, consider these two scenarios: one in which creative is only replaced after its performance has degraded sharply, and one in which creative is replaced more quickly.

It’s common for advertisers to be too reactive with this dynamic: they wait until they see an ad creative’s performance start to decline before they begin work on new replacement creatives. This reactivity necessarily results in a lower level of mean ad performance, and it is especially common when an advertiser finds a creative that outperforms.

This peak-and-valley dynamic creates volatility in advertising performance and thus new user acquisition, app DAU levels, advertising expenses, etc. — all things a company should want to be predictable and calculable. It’s easy to see how a creative process that maintains average performance by replacing aging, deteriorating creative with new, tested, performant creatives helps foster stable growth for a company.

Framing a creative process

Before examining the atomic units of a creative process, it’s valuable to consider what such a process is expected to achieve. In order to regularly produce one performant ad creative, an advertiser needs to walk through a series of six steps:

1. Concept the creative. The advertiser needs to come up with a narrative for the creative: what will happen in the creative, what is showcased in the creative, etc.;
2. Draft the concept. This step is optional: depending on the complexity of the concept (eg. a live action ad requiring hiring actors, writing a script, etc.), the developer might create a storyboard of the creative to serve as a guide during production;
3. Produce the creative. The advertiser produces the creative deliverable, be that a video ad, static banner, full-screen static interstitial, playable ad, etc. This production stage also includes producing variants of the creative concept;
4. Test the creative. The advertiser spends a budget against the delivered ad creatives sufficient to determine if they are at least as performant as the average portfolio of ad creatives;
5. Deploy the creative. The advertiser adds the new creative into its portfolio of live ad creatives;
6. Monitor the creative. The advertiser monitors the performance of the ad creative in live campaigns.

Once an advertiser has decided to invest requisite time and resources into rapid creative production, iteration, and experimentation, it needs to formulate some workflow structure that facilitates those six steps. A creative ideation and testing process — “the creative process” — might look different from advertiser to advertiser depending on team size and internal capabilities, a company’s appetite for outsourcing, the amount of money a company is spending on direct response marketing, etc. But I think any good creative process features at least four traits:

1. Regularity. A good cadence for a creative process is weekly; any longer than that and the turnaround times on creative and the lag between concept ideation and testing get too long to be useful;
2. Accountability. Each person involved in the process should have a clearly-defined role in it, and everyone in the creative working group should be able to depend on those people to complete their assigned tasks;
3. Predictability. The creative working group should establish a specific number of creatives to be delivered on a regular time interval. This specificity is helpful for alleviating the peak-and-valley performance problem described earlier, and it also acts as a forcing function for producing enough creative concepts to regularly produce tested, validated ad creative.

Regularity is incredibly important in this keeping the creative process optimally productive: it’s easy to perceive the immensity of the time scale involved in producing a single creative from looking at the diagram of the creative process steps. For example, a high-quality video ad creative that only utilizes in-app and stock footage (ie. no new footage needs to be filmed) could possibly take one full month to circulate into live campaigns, from concept development to production to test to full deployment (video adds consisting only of in-app and stock footage generally aren’t storyboarded):

This sense of timing can help an advertiser to plan the number of creative concepts they need to produce each week in order to satisfy demand for replacement creatives. The best way to come up with this number is to work backwards up the funnel from the number of ad creatives that are being retired each week. If the advertiser wants to simply maintain performance and replace ad creatives when they reach a certain level of performance degradation, then they can use their “hit rate” of creative performance — that is, the percentage of creatives that prove in test to be worthy of being deployed into active campaigns — to get from the number of creatives needed each week to the number of concepts that need to be produced each week.

As an example: imagine an advertiser is retiring four ad creatives per week. Each of the concepts it generates gets produced as a set of five ad creative variants, and each of the ad creative variants it produces go into a test. If, on average, one out of every five ad creatives it produces performs well enough to be deployed into active campaigns (eg. Hit Rate = 20%), then the advertiser needs to generate five concepts ( 5 Variants per Concept / 0.2 Hit Rate = 25 ad creative variants) each week in order to satisfy its demand for ad creatives.

But even if the advertiser undertakes the task of generating five concepts per week, its first set of ad variants to be deployed might not be ready for a month or longer. This is why regularity is so important: an advertiser can’t quickly produce creatives to meet an immediate need, so it must consistently operate this process in order to pre-empt future performance degradation.

Undertaking the work

With the general form of a creative process defined, an advertiser must find a way to work its own team and resources into a routine. As stated earlier, this is usually weekly: given the long production and testing cycles for ad creative and the constant need for new ad variants, it makes sense to run a creative process on a weekly cadence.

This usually starts with a Weekly Meeting (usually on a Monday) in which representatives from the product, user acquisition, marketing art, and sometimes the analytics team discuss plans and ad creative needs for the upcoming week. Note that this is a cross-functional meeting and should exist independently of any meetings those specific teams might have on their own (eg. a daily user acquisition meeting or bid review). The purpose of this Monday creative meeting is to a) identify creative needs (eg. is more ad creative needed because the budget is increasing? is specific ad creative needed for an upcoming in-app event, promotion, stylistic overhaul, etc.?), to b) get an update on the creative that is currently in production, to c) review and provide feedback on creative that has been delivered, and to d) review and prioritize proposed new creative concept ideas.

In the meeting, the cross-functional team will:

1. Identify creative needs. The creative team should be apprised of the general demand over the next few weeks for new ad creative from the product and user acquisition teams. If, for instance, the user acquisition team will be running a re-engagement campaign in two weeks’ time that requires new, custom ad creative, that request would be added into a creative backlog during this meeting and prioritized accordingly. The product team and user acquisition team should be leading the discussion around how changes to the product or marketing budget will create the need for new ad creative;
2. Get an update on creative currently in production. Usually the art director or marketing creative director will provide the team with an update on where certain creative concepts that were already approved are in the production cycle. Oftentimes, some set of creatives have been outsourced to an agency or to multiple agencies, so it helps in the meeting to get a sense for which deliverables are on schedule. If creative deliveries are understood to be delayed, the team can react accordingly and re-allocate resources to other concepts or move new concepts into production;
3. Review and provide feedback on creative that has been delivered. If the creative process is functional, the team should be receiving new, finished ad creative every week on a rolling basis. In this meeting, the team should review the creative that has most recently been delivered and determine if it is ready to be tested or requires changes (any changes should be noted as action points in the meeting and resolved by the marketing art team afterwards);
4. Review and prioritize proposed new creative concept ideas. A team might have an open document (eg. Google Doc or Airtable table) with concept ideas that the creative team submits on a rolling basis, or they might discuss new ideas in this meeting (I prefer the former). These ideas should be regularly groomed and prioritized for production based on creative needs. I generally think it’s a good idea for the grooming / prioritization work to not be done by committee and rather managed by someone who completely owns the creative process and has the authority to approve / reject concept ideas (usually the CMO / VP of Marketing).

Members of each of the functional teams should walk away from this meeting with specific, explicit action items:

• User Acquisition Team: The UA team should know which ad creatives have been greenlit for test and should begin the process of testing them;
• Marketing Art Team: The Marketing Art team should know which changes need to be made to delivered creatives and which creative concepts have been greenlit for production. Additionally, the Marketing Art team should know which current production processes need to be sped up (if any) to meet demand deadlines;
• Art / Creative Director: The Art or Creative director — the person who runs the creative process — should have a sense of how many new concepts need to be generated based on the current size of the backlog.

Clarity of expectations is critically important here: the purpose of this cross-functional team is to bring utter transparency to each team’s needs and resourcing so that everyone is aligned around what is possible and on what timeline. And the entire point of running this meeting regularly is to try to anticipate future need in a way that eliminates surprises. Marketing should be predictable and boring: if the creative team is operating functionally, short-term scrambles and frenetic thrash can mostly be avoided.

With each team taking its cues from the weekly meeting, they can get to work on their individual contributions to the overall creative effort. For the user acquisition team, this means testing the new creatives that were most recently delivered and evaluating the performance of live creatives. Creative testing strategy and measurement goes far beyond the scope of this article, but it is captured in a full content module in the upcoming Modern Mobile Marketing at Scale workshop series. The content module in the workshop includes a framework for running creative tests and practical reporting / analytics tactics to use in selecting and promoting winning creative into live campaigns (as well as benchmarks for retiring creative).

After the weekly meeting, the art team should set about either producing the agreed-upon creative concepts and variants in house or sending that work to an outsourcer / agency. In general, it is beneficial for advertisers to outsource at least some of their creative production to agencies — having multiple teams working on marketing art contributes to the overall diversity of ad creative styles and adds depth to experiments. Also: maintaining a large in-house art team is expensive and the carrying cost prevents an advertiser from remaining nimble around overall marketing spend.

As marketing art is delivered, the art team should be sharing that creative with the creative team via a shared drive and updating the process document with links to completed assets by format. It can be helpful to have the marketing art team send out a weekly review update (usually on Fridays) to everyone involved in the creative process with updates on the art that was delivered that week so that everyone is aware of what is available for use. Some teams put new creatives into test as soon as they are received, and some wait until the following Monday meeting to review collectively first. How this is approached tends to depend on how closely the CMO / VP of Marketing wants to exert control over what ads get exposed to potential users.

Putting it all together

It can be difficult for smaller companies that are operating their marketing in an unstructured, reactive formulation to adopt a rigorous, forward-focused process — the routine can feel stifling, cumbersome, and overly administrative. And to be certain, implementing any sort of process does come with administrative overhead; in the case of a regular marketing creative review and production process, an entire creative tracking system and testing procedure needs to be created.

But there’s no other way to achieve user acquisition performance at scale on mobile. The process creates the stability needed to consistently test new ideas, and testing new ideas (and exploiting the ones that work best) is the way that advertisers grow spend and maintain performance in the new, algorithmic advertising paradigm.