In a lot of ways, machine learning is magic. It’s incredibly impressive what artificial intelligence has been able to accomplish lately. (Insert gushing fanboy examples here). However, it’s also true that data science buzzwords such as “machine learning” and “artificial intelligence” are overused to the point of absurdity.
Go to any marketing agency’s website, and you’ll see it littered with references to “advanced AI” and “machine learning everywhere!” (looking at you, Xaxis). Entirely absent is any mention of how the agency actually does that – it’s proprietary, of course! The reality is having algorithms inform marketing strategy is table stakes and trivial solutions can be sold as advanced AI.
So what I want to share is a few tips on when to be suspicious about a marketing agency’s artificially intelligent machine-learning super deep learning platform called BoB.
Tip-off #1: Unspecified Usage of Deep Neural Networks
Deep neural networks (DNNs for short) is the reason for all the recent hype with data science and machine learning. At the risk of repeating myself, the applications of DNNs in computer vision and text analysis have been super impressive. But for them to be effective, one needs massive amounts of data, and not just a lot of rows (for example, the billions of impressions Goodway serves daily) but also a lot of features (for example, the brightness of an individual pixel in a high-resolution photo).
While ad agencies definitely have the volume, we don’t have the high dimensional feature space – we’re mostly working with a handful of attributes about a user like browser and device type.
That’s not to say I doubt ad agencies are actually building and using DNNs, just to say I think a simpler solution would have sufficed, and the DNN isn’t actually getting you anything other than a data science buzzword to confuse non-scientists.
Tip-off #2: Extraordinary Results With No Mention of Testing Methodology
No judgement, testing is super hard. The increasing “walled garden”-ification of the ad tech space makes it even harder.
While a straight AB test is nearly impossible for anything multichannel, we still have geo-based tests that can be quite effective. The red flag is when agencies report results of a pre- versus post-period with no accounting for seasonality. For example, the number of gym sign-ups increases at the beginning of the year – that’s New Year’s resolutions, not marketing.
All that’s to say is if agencies went through all the trouble of building a machine-learning algorithm and putting it into production, then they hopefully went through the trouble of adequately testing it too and should be excitedly mentioning the rigor of the testing methodology. Otherwise, I’m going to assume those numbers are baked to perfection.
Tip-off #3: Anything Broadly Labeled “Fraud Prevention”
No, not because there’s no fraud – there definitely is – but because fraudsters are sophisticated nowadays and simple solutions are wholly insufficient. A friend of mine said her agency blocks a handful of IP addresses that are marked as “suspicious” by an arbitrary metric. They do this in the name of fraud prevention and then sell it as such. 😱
Meanwhile, actual fraudsters are getting around the fraud prevention from companies dedicated to this(!). That’s not to say fraud prevention software is junk. The stuff from companies like Integral Ad Science (IAS) is legit. But I’m suspicious of a marketing agency doing it because it’s probably something very simple they’re trying to sell as sophisticated.
We’re Better Than That
For the record, I don’t think marketing agencies are out there throwing data science buzzwords around to deceive or swindle you. The more likely scenario is the team writing the content isn’t the team building the AI. Above are three easy ways you can tell there’s been a drift between the marketing materials and reality.
Goodway Group is different. The data science team (which I lead) works closely with the marketing team to make sure our marketing materials match our algorithmic reality. Case in point: I tried to get away with just ranting about algorithms and data science buzzwords in ad tech, and the marketing team insisted I write about what distinguishes Goodway Group. That’s to say, Goodway Group cares deeply about being honest with our clients, so much so that it’s ingrained in our company purpose: to be the beacon of honesty and intelligence in marketing.
Goodway has an algorithmic suite for optimizing our client’s marketing objectives too. No, it doesn’t use deep learning, but it’s sufficiently complex that I can’t adequately describe it at the close of what’s supposed to be a short blog post. I promise to write another blog post about our sweet algorithmic suite, and when I do, I’ll link it here.
I’m Lucio Tolentino, head of data science at Goodway Group, and I approve this marketing material.
Lucio Tolentino heads the data science team at Goodway Group. He has a Ph.D. in computational epidemiology, which he uses to build optimization algorithms and inform proper testing methodology. Beyond applying scientific principles to the complex ad tech ecosystem, he enjoys riding bicycles and other outdoor activities.