This weekend, a “glitch” (or choose your synonym: bug, error, irregularity, etc.) caused Meta’s ads platform to behave unexpectedly. I heard of unusual performance from several portfolio companies on Sunday morning, and I also began to see frantic statements from digital advertisers on Twitter. This is the statement that Meta sent to advertisers regarding the issue:
Given limited communication from Meta, it’s not possible to ascertain what, actually, went wrong. From what I can tell, campaigns for both app and web destinations optimized to purchase objectives saw CPMs and delivery jump precipitously starting early in the morning, Pacific time, on Sunday. My own theory is that some sort of conversion measurement or modeling issue caused Meta’s ad platform to miscalculate the value of the impressions it was delivering to advertisers, which resulted in delivery being accelerated. Rishabh Jain, whom I recently interviewed on the Mobile Dev Memo podcast, disagrees and thinks it was either a budget pacing issue or a problem with an Advantage Shopping preferencing schema. I doubt we’ll see a post-mortem from Meta about the issue; we’ll likely never understand what truly happened.
Some advertisers reported that this issue depleted their entire daily budget in just a few hours at performance levels (eg. ROAS) far below historical standards. Many performance advertising operations optimize for thin margins on maximal ad spend; even a single day of inadequate performance could produce severe consequences for a scaled advertiser’s business. One day is 3% of a month; I’ve seen advertisers reporting near-0% ROAS for their Meta campaigns on Sunday, with budgets having been fully exhausted.
Meta has been relatively quiet about the situation. Aside from an update on the Meta Platform status site, a tweet from a member of Meta’s ad platform team, and a short response to an inquiry from AdExchanger, I haven’t seen many official statements from the company about the issue. It remains to be seen whether refunds will be issued to advertisers; calculating what each advertiser is owed seems incredibly challenging. Every advertiser affected is owed something, but advertisers were impacted to varying degrees, and the counterfactual — how much would this advertiser have spent absent the glitch, and at what standard of performance? — for each individual advertiser doesn’t strike me as realistic. I wouldn’t be surprised if Meta simply issued credits to advertisers.
What I think this episode highlights is the extreme risk inherent in marketing automation, and why total marketing automation is unwise. As I write in Exoskeletons, not cyborgs:
Second: media spend represents a considerable expense for many digital advertisers — in some cases, the largest single expense line item. Real money is at stake with digital advertising, and advertisers should be reluctant to surrender control of that spend to a system that could very well make mistakes. Some automated data products, like recommendation systems, have relatively contained downside risk; advertising systems are fragile and comprised of many components, and an edge case or unforeseen feedback loop in an automated advertising process could cost an advertiser an unacceptable sum of money.
The tail risk for an advertiser in an algorithm malfunction could be catastrophic. Not only is performance advertising the single largest non-labor expense for many companies that build digital consumer products, but it’s also often the primary or sole driver of revenue generation. If an ad platform breaks, even for a day, the potential for loss is immense.
Meta’s CTO acknowledged this month that the company will begin commercializing generative AI, possibly for creating advertising assets on behalf of advertisers, this year. I make the argument in Abandoning intuition: using Generative AI for advertising creative that Generative AI unlocks more value in the ad creative conception and ideation process than in the asset production process. But the Meta irregularity from this weekend highlights a risk in wholly automated advertising creative production: that the incentive structure for ad platforms is simply too misaligned with the best interests of advertisers to empower those platforms to manage the digital advertising process end-to-end.