Podcast: An oral history of mobile user acquisition (with Nebojsa Radovic)

On this week’s episode of the podcast, I am joined by my friend Nebojsa Radovic, the Senior Director of Mobile User Acquisition at Zynga and a battle-hardened mobile marketing veteran. In the course of our conversation, we discuss the evolution of mobile user acquisition from the era of manual campaign configuration to the contemporary operating environment of automated, AI-driven workflows. We reflect on how the industry has shifted from highly technical, high-headcount operations toward lean, specialist teams focused principally on creative optimization and signal engineering. Among other things, we cover:

• How AI-driven automation will permanently redefine the technical requirements for marketing roles
• Whether the consolidation of major ad networks will eventually stifle innovation within independent mobile gaming studios
• Why high-volume creative production has become the primary lever for performance in an algorithmic buying environment
• If traditional media buying expertise is becoming obsolete in the face of sophisticated signal engineering strategies
• What future organizational structures will look like as marketing teams become smaller and more specialized
• When the industry will reach a saturation point regarding the effectiveness of cross-channel attribution models

Thanks to the sponsors of this week’s episode of the Mobile Dev Memo podcast:

• ⁠⁠INCRMNTAL⁠⁠⁠. True attribution measures incrementality, always on.
• ⁠Branch⁠. Branch is an AI-powered MMP, connecting every paid, owned, and organic touchpoint so growth teams can see exactly where to put their dollars to bring users in the door and keep them coming back

Interested in sponsoring the Mobile Dev Memo podcast? Contact Mobile Dev Memo advertising.

The Mobile Dev Memo podcast is available on:

• Spotify
• YouTube
• Apple Podcasts

Transcript

Eric Seufert: Welcome to the Mobile Dev Memo podcast. I am your host, Eric Seufert, and I am very excited to be joined today by Nebojsa Radovic. Nebo, welcome to the podcast. It has been a long time coming.

Nebojsa Radovic: It is an honor to be here. We have known each other for over a decade and worked together, and I am really glad to spend some time with you talking about the industry and the recent trends.

ES: I will let Nebo introduce himself, but I would describe him as a grizzled veteran of mobile gaming user acquisition. He has been in mobile gaming since the dawn of the category and worked at some of the most cutting-edge mobile gaming studios in terms of how they approached UA. He is perhaps the House MD of mobile UA, having seen everything. We finally got the green light to get you on the podcast. Why don’t you introduce yourself to the audience in your own words?

NR: At this point, everyone knows me as Nebo. I moved to the US in 2012 after studying computer science back in Belgrade, Serbia. I started my career in mobile gaming in 2013 and have been in the industry ever since. I worked at all kinds of different companies of all different sizes, starting my career at Nordeus, which got acquired by Take-Two, and continued my career with Machine Zone, which was likely the largest mobile gaming advertiser at the time. I then continued at NetEase, where the two of us had a chance to work together and had a wonderful time there, and now for almost the last six years, I have been with Zynga. I have seen developers of different sizes, all in mobile gaming, and obviously different eras of user acquisition.

ES: When I was thinking about how to structure the podcast, I was thinking about what I would ask someone who has been in the space since the inception. I think one really interesting perspective to get is a history of mobile UA from you. Talk to me about mobile gaming UA in the Machine Zone era. At one point, they must have been the largest marketing spender in the gaming category, but I think at one point they were the largest spender on Facebook.

NR: It was a massive advertising operation at the time with hundreds of millions of dollars in annual spend and two or three Super Bowl commercials. It was a pretty remarkable point in time and a pretty remarkable team. It was a very different era as well. Media buying at the time was super manual. It is hard for people who started their careers post-COVID to understand how manual things were. We had to manually pull daily spend reports and stitch them together because there were no reporting APIs. Creative reporting was pretty clunky at the time as well. Having access to certain data and better reporting was a true advantage in the market.

If you could get creative reporting or publisher-level reporting, where we could target a specific publisher, let’s say a solitaire game or the number one downloaded game on the App Store, that would really give the team a competitive advantage. One thing about Machine Zone that has not been talked about much is that Machine Zone really helped push the ecosystem forward. A lot of these things like reporting APIs and creative reporting came to life because Machine Zone was a big, pushy advertiser asking for these different features. As a result, the industry managed to capitalize and become more sophisticated in media buying compared to today.

It was also a really big team. It was over 100 people managing media buys, which is crazy to think about from today’s perspective where teams are generally pretty lean, especially on the user acquisition side. On the flip side, there were teams who were super specialized in specific channels. It really felt like being part of the UA all-star team where you would work with the person who is the best in the world at managing Google or Pinterest because our budget was pretty big and we could test new channels even if that meant those channels would not be immediately profitable.

ES: Talk to me about the structure. How was that organized? You would have channel-specific people and pods. How did that all coalesce in terms of reporting and in terms of managing those pods, and where did this all flow up into?

NR: The Google team and the Facebook team were pretty big, anywhere between five and ten people managing each. It is important to remember this is the pre-UAC and pre-Advantage+ era, so it was pretty manual. On Google, you had web-specific campaigns, keyword-specific campaigns, and then YouTube-specific campaigns when YouTube became a thing on the advertising side. These were highly specialized teams with highly customized reporting because Singular was just becoming a thing at the time for cost aggregation. To get this pub-level or placement-level reporting from Facebook or Google, you had to build a customized report. That was a real advantage because most small advertisers didn’t know about it or have access to it. That is why having such a big team helped so much for Machine Zone to stay ahead of the curve.

I focused almost solely on DSPs and programmatic buys. When you are such a big advertiser and you work with multiple DSPs and have pub-level reporting across the board, you can see the most efficient ways of buying from a certain publisher or a certain app. Even nowadays, it is very sci-fi because we don’t really get this level of insight anymore. At the time, because Machine Zone was so big and had so many people who could build custom reporting and focus on things such as the most efficient way to buy from a solitaire app, you could do things that no one else could do. Sometimes having more people is actually helpful because you can do things at a higher level of granularity with more precision than others. Obviously, that is changing now with AI, but at the time, that was a real advantage because we just had access to more data and more people to actually process that data and make better user acquisition decisions.

ES: Talk to me about that because I think that is worth going into some detail on. When you say the best way to buy from a solitaire app, you are talking about app-level reporting, but what you are talking about is much more specific than that. You wanted to get the third impression in a session in placement X in that solitaire app.

NR: The third impression on the banner placement in the solitaire app in Germany on iOS 14.4 versus the second in the US on Tuesdays. Because you were spending so much and had so much data, you kind of side-stepped the curse of dimensionality in a lot of ways. Because you had all this reporting, you were able to get enough data to look at all the combinations of things and say this is when that third impression in Germany on Tuesdays on this iOS version is going to be more profitable than the second or the first because the first is more competitive. You had that across the board and were operating against that for X number of apps.

These had to be large DAU apps because you needed enough data to make proper decisions. For those who don’t fully understand how those apps are monetized, they don’t use just AppLovin for monetization; they use ten different networks in a waterfall or header bidding where they dynamically bid at the same time. Maybe it is cheaper and more cost-efficient to buy an impression from Unity than AppLovin. We had to figure out and reverse engineer where the margins are and try to find a more efficient way to buy. At the time, that was called supply path optimization. It is not a term that is thrown around much nowadays, but it was really powerful. This also meant that we could strike deals with publishers and maybe buy directly in case they were big enough and their inventory was valuable enough to us.

Nowadays, I don’t think anyone does it, especially at scale, but at the time, that was a very successful strategy. That led us to the next stage of UA, which was acquiring DSPs and building first-party DSPs where you wanted this manual logic to turn into algorithmic bidding, which is essentially what started happening after 2015.

ES: SPO is not a term that gets used a lot in mobile, but if you go to a Marketecture event, you will hear that from every second speaker. That is a programmatic web concept, but it is still very common to hear that. Talk to me about who the people were producing those reports. Machine Zone would hire a machine learning team and they had economists who would do that kind of analysis and produce that kind of reporting. It was a big team across the board, not just on media buying, but analytics and data science were really strong.

NR: There was a dedicated marketing technology team. It is not that you just had to aggregate all the data and build reports; you also had to build tools for the team to operate better, like campaign naming tools, creative naming tools, or renaming tools. Any sort of automation was based on the saying at the time that if anything takes you more than 60 minutes, it should be automated. People don’t understand how forward-thinking this was at the time. Now it just makes sense, but because there was a marketing technology team who built these automation workflows and tooling, we were ahead of the curve and moving much faster than most of our competitors. We were able to spend significant amounts of dollars on games that were not necessarily for large audiences. These games were pretty hardcore, but we found ways to find these high-value customers.

ES: What was the channel landscape like at the time?

NR: In 2015, I believe that Facebook and Google were around 60%. Facebook was still pretty dominant because it was a large DAU app and the only mobile-app-first media buying solution. This was pre-UAC where Google was still fairly manual. Google was probably in the 20% range, and DSPs or programmatic buying was pretty big at 20% to 25% of the overall spend. The reason I say programmatic buying is that essentially all the video networks and ad networks nowadays, such as AppLovin and Unity, but also Liftoff, Vungle, and IronSource, were separate networks then. You could either buy directly by working with a company like IronSource or you could buy programmatically through a DSP. We bundled all of these together because it is the same type of buying. Some percentage of spend went to incentivized traffic, which is known as CPE right now. Pre-loads were also probably in the 5% range with companies like Digital Turbine and PinSight.

ES: That is where we first met in person, right? It was the infamous Mobile Growth Summit Berlin 2015 when you did the stand-up and we had a wonderful time. What has gotten easier in mobile gaming UA since then? You are describing the most technically savvy mobile gaming UA operation with 100 media buyers and a large foundation of analytics infrastructure that gives an edge and competitive advantage to a company that can afford and support that. But that is not in the best interest of a Meta or a Google. They would like to provide that same level of sophistication to everybody, so they should build those tools. That is largely what happened. We had UAC, which was Google’s end-to-end automated product for mobile app installs, which predated Advantage+ and PMax and all those other systems.

Talk to me about that evolution where Google and Meta and all these other companies wanted to bring that level of sophistication to their advertisers to give them the benefit of those tools. Now you would probably never find a team that big at a mobile publisher. What has gotten easier, and conversely, what has gotten harder? Those bring challenges themselves, so in terms of just spinning up a mobile UA team, what can I do now that would have been unthinkable ten years ago versus the challenges I am going to face that are totally new or a lot more acute now than they would have been back then?

NR: When UAC was introduced, no one thought that would work. Everyone thought this makes no sense because we want all those manual levers and we want to be able to target specific publishers and specific keywords. Honestly, for the first two years, UAC was not particularly performant. But over time, they figured out how to find high-quality customers and introduced more sophisticated ways of bidding. Most importantly, they democratized access to sophisticated UA tools. That is when the playing field got leveled and more advertisers could spend significant amounts of money on Google UAC and then Facebook Advantage+.

You didn’t really need to be this highly technical, highly sophisticated media buyer. Things got way easier on the touchpoint side of things where you didn’t have to do as many manual, grunt-work things to manage campaigns. But that also meant more competition. More competition meant that more companies could spend money at scale on these channels. One thing that made this situation even worse is consolidation. IronSource and Unity are becoming one company, Liftoff and Vungle are becoming one company, and AppLovin is acquiring MoPub, which was the largest mobile advertising exchange. Instead of working with 30-plus partners where you need a large team to support that, now you have four or five different channels that have 95% coverage.

You really just need people on the team who understand these five channels really well and you’ll get what you need instead of trying the shotgun strategy of working with 20 channels. Consolidation is a starting point, and fewer channels make it slightly easier to manage user acquisition efforts. The flip side is that most of the buying nowadays is algorithmic. You don’t move all those knobs and play around with different levers; the algorithm is doing the buying on your behalf and you are essentially in charge of four things: bid, campaign type, budget, and creative.

Creative remains the only high-touchpoint item that you can optimize and use to impact the actual performance. Campaign execution is easier, but hitting your goals is much harder because competition significantly increased. One more thing that happened with consolidation that I didn’t find many articles writing about is that the access to Western audiences is significantly democratized. If you are a Chinese developer who has a largely successful 4X game doing really well in China, for you to succeed in the Western market, you just go to AppLovin, give them all of your money, and they’ll bring as many high-quality players into your game as possible. This wasn’t really the case ten years ago. Competition opened up and grew in size to be truly global. Because all of the buying is algorithmic and on a handful of channels, it’s extremely competitive. It’s not necessarily easy to launch games and run user acquisition efforts profitably anymore. That’s the flip side of the democratization of media buying.

ES: I think the way I characterize it is that you can’t win by throwing more bodies at the problem anymore. The challenges have gotten a lot more data-science-oriented. In terms of running your business, they’ve seeped a lot more deeply into the product, which maybe is a good thing. That just forces everybody to make better products and really compete there. If you don’t have to compete at the UA edge because everyone’s there using the same sophisticated channels, then maybe it’s just about building a great consumer experience.

NR: Exactly. UA is so expensive and your business is so dependent on it because organics are no longer a significant contributor to the overall business. You really need to make sure the products you’re making will work on these channels. If your game is not able to successfully run AppLovin campaigns at scale, it’s highly unlikely you’ll have a successful product. This informs the product development cycle. You have to test these channels in early product development cycles to see whether there is a product-market fit or you need to change something like gameplay or the art style in order to increase your product’s chances of success through lower CPI or higher average revenue per user. Most of the teams that are doing user acquisition successfully nowadays and spending at scale are the teams that run campaigns for the best products. It sounds logical, but it wasn’t the case a decade ago where there were still some opportunities for arbitrage. Truly, if you look at Royal Match or Monopoly Go making over a billion dollars in revenue a year, those are truly amazing products with best-in-class retention and monetization profiles. UA is just amplifying great products and putting fuel on the fire.

ES: What is the process? With all this automation, what are you doing? What does your week look like and how do you structure your process?

NR: When you are leveraging algorithmic platforms, what is being fed into that algorithm becomes the priority. We already talked about creative. You really need to think about how different creatives perform and what the retention profile is for a certain group of creatives. Also, one of the key topics is signal engineering. It’s not a new topic; we worked on this at NetEase. It’s essentially what type of signal is being fed into the platform’s algorithm. I’ll give an example of AppLovin because it’s probably the most relevant for the mobile gaming crew. You need to drive 20 purchases a day at a campaign level in order for the campaign to exit the learning phase and start performing well.

How do you get to those 20 purchases a day? What is the budget that is needed? What is the starter pack in the game so it can actually happen? If you have a $20 starter pack, maybe your conversion rate won’t be high enough to get there. This is essentially why it’s important to work closely with the product team to explain why certain numbers of purchase events and certain conversion rates are important for the campaigns so we get enough signal for the campaigns to exit the learning phase and start performing. That’s one side of the story, signal engineering and understanding how many purchases are being sent out every day if you’re optimizing toward a specific revenue target.

What’s the daily average revenue per user on day zero or day seven or day 28, which is the dominant conversion window nowadays? The second side of the story is creative. Being such a huge lever, it can either help you lower the cost, which means that you can drive more purchases for the same budget, or it helps with audience discovery. Again, working in tandem with the product team, especially for games that are quite mature—the games I am working with are ten-plus years old—you’re trying to find different audiences that are truly incremental to your game or your product. Creative helps you lower the cost or find different audiences that will have higher ARPU and in that way hit the campaign goals.

The human oversight portion of the job becomes really important because you need to make sure you understand how these different things interact and how they impact the overall campaign performance. You need to coordinate with the product team across a portfolio like Zynga’s, which is a lot of products to be coordinating with. Depending on the budget size, if you have a smaller budget, it takes more days to hit those numbers and exit the learning phase. That will inform your cadence; maybe you won’t make daily changes, you’ll make weekly changes.

Depending on the life cycle of the product, if it’s a new product, you’ll meet with them weekly and talk about the most important milestones, opportunities, and gaps to try to improve our return on ad spend or overall product-market fit by changing things in the game. Maybe minigames we are introducing are just not performing well. There is definitely that component of working very closely with the product team to inform them what we need in order to succeed. Outside of the product team, it requires a lot of coordination with the UA team itself managing campaigns and then with the analytics team trying to understand how different campaigns perform and what is the LTV of those users. The LTV of a user coming from a return on ad spend campaign versus a user coming from a blended ROAS—by which I mean we blend in-app purchase and in-app ads revenue—is going to be very different. It’s a different profile of a user and we have different LTV curves to estimate.

You have to work very closely with the marketing analytics team and the creative analytics team. Different creatives drive different user types and we need to understand what their LTV curves and monetization profiles look like. We then work closely with the creative strategy team to make more of those that work and fewer of those that don’t. It requires a lot of coordination, and then multiply that times eight or 25, which is the size of the Zynga portfolio, and you get a pretty complex operation.

ES: Let’s delve into that creative process. Talk to me about how you do it and what the goals are. Are you targeting a volume of output? I imagine at a company that is publicly traded, there are brand guardrails or constraints around what kind of creative you can launch. You have to navigate that tension that can be difficult to communicate, which is that if we show this kind of a creative, we’re going to get this kind of an audience and this audience is going to behave in a different way. You have to segment these people out and think about them as groups. How do you go about producing the creative that you use?

NR: Different games and different budgets have different creative requirements and refresh rates. On games that spend a few hundred thousand dollars a month, you maybe refresh creatives once a month. On games that spend millions of dollars a month, you have to refresh creatives almost daily. One thing that changed is that volume requirements keep increasing. The top spenders on AppLovin introduce thousands of ads a month. Arguably, creative teams are much bigger than UA teams nowadays. If you talk about these large AppLovin advertisers, they have 300 to 500-person creative teams because creative volume became a big thing. The platforms claim they know which creative will work even before they serve one impression against those creatives.

The volume imperative became a big thing and we have to try to produce more. What is the success metric for creative? It’s hit rate, essentially the percentage of media spend. Some teams call it hero rate. If you build new creatives and those creatives are able to spend a sufficient amount of money, let’s say 10% or 20% or 30% of the overall share of wallet or overall spend, then those creatives are considered successful because they’re able to hit specific ROAS goals. The goal of a successful creative production process is to drive more heroes, which is essentially having this setup where there are creative strategists, creative analysts, and producers who make sure that ads get produced. They work closely with the user acquisition team to understand what’s working and what’s not working.

There’s also a separate effort looking into what is working for our competitors and what are the most popular trends on social media and on intelligence tools like Sensor Tower. That informs the creative production roadmap and we start producing those assets. You brought up one more point which is really important: what happens when you work with different licensors and IPs? That does complicate the process and slows it down because if it takes three to four days to create a video ad, it also takes another week to get approvals. If you’re working with licensors, things will take longer and they might not be okay with certain things, for example, misleading ads. They might not be okay with you branching out of a specific world. If you’re a Harry Potter game, you have to stay within a specific Harry Potter world and art style. There are some limitations and you have to understand those before you branch out and start producing ads. Over time, we got really good at this. We’ve been working with these licensors for a very long time and we know what to make and what are the safest ways to build creatives that will perform.

ES: What did it look like at Machine Zone without modern AI tools? How were you making the creatives and how many were you making? There was probably a more diversity of type; you were working with banners and keywords. What were the major differences between then and now?

NR: Nowadays it’s mostly video and playable or interactive ads. Back in the day, the team size was around 75 people, but a lot of creative development was automated. Localization was fully automated and most of the static image creation for different sizes was automated through scripting tools. There was outsourcing and there was internal development. For anything that had to do with TV, like working with Arnold Schwarzenegger, Kate Upton, or Conor McGregor very early on before he was a big shot, there was a professional team to do that. It wasn’t everything done in-house. Gabe Leydon was really a visionary and he once claimed that we produced 200,000 ads in a given year. The most infamous ad for Machine Zone was that famous playable that probably was the most played game in the world at the time. It did really well because Machine Zone thought about IPM and retention super early on. The way creative was made and run was super data-driven and we had this closed loop between creative strategy, creative analytics, user acquisition, and creative production.

ES: You talked about the pods, like the Facebook pod and the Google pod. Is that how you structure teams today? What’s the right team structure in the current environment and what are the roles that exist on a mobile UA team? Media buyer as such still exists, but certainly not hundreds. Who’s populating these teams?

NR: Especially now with AI as an efficiency booster, you need fewer people to do more things. We have a structure of three pillars: media buying, creative, and analytics. You need people who can fit into each one of these pillars and understand how they interact with each other. For media buying, you need more of these T-shaped, AI-fluent UA leads who can run UA for one game on their own. You don’t really need to make changes every day on Facebook and move things around; you make changes every other day, so in terms of actual grunt work, it’s much easier. But you need someone senior who understands AI and how to harness the power of AI to get the most out of the tools and products that we have.

The goal right now is to build a team of super ICs who can do this, but also to build systems that help them do their job extremely efficiently with the help of AI tooling. The second part is creative, where creative producers, creative strategists, and user acquisition folks work together to understand what needs to be made, whether it is video or playable, and whether we need to localize or culturalize it. This is still fairly manual because we don’t know what’s going to work even though there’s so much data. It’s all about trusting the process and creating a sufficient amount of creative until you find the next hero. The job of a UA person or UA lead is to work closely with creative strategists who sit as a bridge between UA and creative production. They understand the UA numbers, retention profiles, and the IPM and CPI correlation and then talk to creative production and marketing artists to develop those ads. They meet weekly to discuss what’s being made and work together on briefs, and then those briefs are turned into actual creative assets.

The last point is marketing analytics, which usually consists of two teams. Creative analytics provides all this creative-level data, and marketing analytics helps the user acquisition team understand what are the opportunities. They work on predictive LTV models or work with data science to build accurate models that help us understand what the performance will look like in the future and help us invest money in the right bets and the right campaigns.

ES: And what about signal engineering? Is that a dedicated role or is that an analytics function?

NR: It’s not a dedicated role; it’s essentially what everyone on the user acquisition side is aware of in terms of purchase or event thresholds. They are the ones who talk to the product team and try to understand what we can do to drive more purchases and if we need more budget. The marketing analytics person can assist if we need some sort of custom reporting. Discrepancies are a big part of our job, so partnering with marketing analytics to understand the discrepancies between partner networks and what we see internally is super important. This is why this thing is not fully AI-automated; there’s just human oversight that is extremely important, maybe more important than ever, because wrong signal can break things. Both the UA team and the analytics team analyze this and try to validate the data and make sure it’s accurate and helping us succeed.

ES: What about channel mix? You mentioned you’ve got the four to five channels that give you 95% of your spend. Back in the 2015 era, testing new channels was 10% of your time because you were always onboarding a new one. I feel like that’s just probably not the case at all anymore. How do you think about that and is there a sense that you need to optimize a channel mix or is it just driven by ROAS?

NR: When I think about new channels and channels in general, I think about three things. One is scale. Does that channel have sufficient scale? Facebook and Google are the highest scale channels. The second one is ROAS or the cost. Can I actually hit my ROAS targets on those channels? Usually, you are able to hit your goals on AppLovin, Facebook, and Google. The last one is effort. If you have to spend six hours a day optimizing a channel, no matter what the ROAS is, it’s usually not worth the effort. I think a lot about the ROI of effort because I made this mistake multiple times in the past. Even when we worked at NetEase, I would spend an X amount of hours optimizing an incentivized network which drives ten purchases a day instead of optimizing Facebook that drives a thousand.

In this new setup where there are super ICs with an AI exoskeleton, you really need to think about where the time goes and if it is spent most efficiently. The most efficient way to spend time right now is to focus on these high-scale, high-ROI channels. The other reason is targeting. We do care about highly engaged, high-ARPU customers, and those are not easy to find outside of the Walled Gardens because they know exactly who’s playing games and who’s spending in those games. Testing that outside of these Walled Gardens means that you have a shotgun approach and it’s not necessarily easy to succeed unless you have a lot of time on your hands to experiment. This is probably something smaller teams do; we mostly focus on high-scale, high-ROAS channels. The only exception for gaming are these pay-per-engagement networks that I mentioned briefly earlier, like Mistplay and Almedia. They’re pretty big at this point and they’re super relevant for the audience, which is why a lot of gaming advertisers focus on those and sometimes spend up to 15% of the budget on CPEs.

ES: Talk to me about the changing team structure and process as you scale up. When you’re spending six figures a month, creative is being refreshed once a month or so, and you get into seven figures and you have to reconfigure the process. How does the workflow and the focus change as you go from six to seven and then seven to eight figures?

NR: My recommendation for smaller companies who are just kicking off is that if you have a six-figure budget, just do Google and AppLovin and do it well. Google on Android is the number one channel on Android and it does really well and is truly incremental. Your measurement also becomes really easy when you have only one channel. Just do Google on Android and AppLovin on iOS and make them work really well. Once you’re ready to scale up, start adding more channels. As you go from a few hundred thousand dollars to $500,000 to a million dollars, you’re slowly adding more channels. You start adding the Unitys of the world and the Metas of the world. Meta is not among the top three channels nowadays, so you usually add it much later in the mix. You can still do that with sort of a fractional team, maybe one UA lead who owns everything, but you have shared resources on the creative side and shared resources on the analytics side.

Things start getting more interesting as you start scaling to millions of dollars a month or tens of millions of dollars a month where essentially you need a dedicated team working on that game. You need multiple UA people and a dedicated creative production team because you need to scale from four creatives a week to 100 creatives a week. That requires a dedicated pod, a dedicated creative strategist, and a dedicated marketing analytics person because it plays such a big role in this phase. On the strategy side of things, as you scale from one million to five million dollars, you start adding more geos. Localization and culturalization start becoming a more important effort. If you’re advertising in Japan or APAC or Middle East North Africa, you need creative resources to actually support that or local agencies who can support you in the culturalization efforts.

As you go from seven figures to eight figures, what becomes relevant is also audience segmentation, trying to find different audiences as you’re sort of saturating the golden cohorts. Brand building becomes more important because you’re spending tens of millions of dollars a month, which is why we start seeing TV campaigns, partnerships with IPs and licensors, and brand ambassadors being attached to the game. This is very common when teams start spending tens of millions of dollars a month. The last thing is that measurement becomes incredibly complex. Last-click attribution will only take you so far, so you really need to think about incrementality, media mix models, or marginal cost. Does adding more channels actually drive more customers or just more purchasers? That’s again why you need dedicated pods and dedicated teams working on one game because the spend level is so high and risk goes up with it.

ES: At that scale, measurement just becomes the job. I get bombarded with all these pitches for these companies that are selling agents to replace your marketing team. I don’t know that we’ll never get there, but I’m always thinking that Meta Ads Manager is an agent and Google PMax is my agent there. For all of these platforms, they have their agent; they’ve built it. I don’t need an agent to coordinate those things. I guess I do, but the reality is that the job is dealing with this consistent stream of edge cases where things break and then interpreting the measurement. No one really knows how to build that from the outside. Even on the inside, it’s going to be very difficult to build a tool that generalizes beyond one single client. That agent you built probably requires human oversight no matter what, and probably just is this assemblage of humans that you’ve got dealing with fires and all the complications of all the things that we just talked about. I’ve got the marketing agent: that’s Advantage+. I’ve got the agent: that’s PMax. What I need is a human to deal with this constant, relentless stream of things that break, and that’s the job.

NR: I always joke that I’m a plumber just dealing with data pipelines and all the breakages. If you use an off-the-shelf solution, everybody’s going to see the same results. How you succeed is by finding non-obvious patterns and exploiting them, like finding different ways to find an audience, whether it was misleading ads or new channels or UGC ads. You have to always try things and not use things that already exist. That creativity part is super important and it will mostly be human-driven for a long time. Our job is to remove the friction. The way I think about AI is to build workflows. You don’t need to upload creative on Facebook anymore; you can build an agent where the producer pushes the ad directly to Meta and then Meta can also directly run those ads. But the pipeline stuff, the data stuff, the breakage—that has to be done by a human and there has to be a strong human oversight who knows what they’re doing and they’re doing it well. That’s how I see the current shift and transition from somewhat AI to a full AI marketing stack.

ES: Talking about that reminded me of Noob vs Pro. Remember that we had this hero creative at NetEase that just carried us for three months? We didn’t come up with the concept, we kind of ripped it off, but it just worked. It was the absolute champion creative that we had and it beat every other creative. We scaled our UA during that period to unbelievable levels of growth and spend, and that was it. It was just having that one creative carry us for that long. Nebo, thank you for taking us on a trip down memory lane. It was nice to reminisce about the frantic 2015 era. How can people hear from you? How can they connect and consume your wisdom?

NR: I’m pretty active on LinkedIn and on X. My username is “eniac” like the first computer, E-N-I-A-C. Find me on X or LinkedIn. You can just email me. Happy to help. Thank you so much for the opportunity, Eric. It’s always great to talk to you, and thank you for a decade-plus worth of content and propping up this industry and making us all better at our jobs through Mobile Dev Memo.

ES: That’s very kind of you. Here’s to another decade. Cheers.

NR: Cheers.