To begin, let’s establish a fact: Machine learning and artificial intelligence (AI) have been booming in academia over the past decade. Consider the following chart, which shows the number of research papers that have been published about AI since 1995 (gray line). But it’s not just research groups and universities. Serious investing groups with serious money are pouring funds into AI-based companies around the United States (red line):
Those B’s are for billions, by the way. Heavy investment in artificial intelligence isn’t going to slow down. It’s only a matter of time until AI is a mainstay in every industry, advertising included.
Machine learning is everywhere around us. Siri? Machine learning. Netflix’s “Recommended” section? Machine learning. The spam filter on your Outlook Inbox? Also machine learning. Machine learning covers a broad swath of mathematical techniques, complexities, and applications. At its core, the concept driving machine learning is fairly simple. The whole idea, from 30,000 feet, is that you can define a problem for a computer, and it can solve it without being explicitly told how to solve it. I usually like to think of machine learning as akin to teaching a child. You might point at a car driving by and tell a child, “That is a car.” You don’t mention why it is a car or what makes it a car, just that it is one. The child might notice that a car has wheels and that it moves on roads. Then a truck drives by, and the child says, “That is a car!” You would say, “No, that’s a truck.” The child might then figure out, on their own, that a truck also has wheels, also moves on roads, but is bigger than a car. A machine learning model learns in exactly this way.
Let’s back up for a second. If you are new to the world of statistics and mathematical modeling in general, you should be aware that there are two main schools of thought: classical statistical modeling and machine learning. One isn’t better than the other – although some of the more zealous members of either tribe might beg to differ. They are simply different. In fact, sometimes the lines get blurry. For example, you can actually use machine learning to estimate classical models. There are some things that traditional approaches accomplish that cannot be replicated with machine learning and vice versa.
Classical statistical approaches were developed in a time when computing power was limited or nonexistent. As a result, the dimensionality of the problem being worked on needed to be reduced. Statisticians tend to use strong assumptions about the behavior of the data that they are considering. Subsequently, some analysis techniques are discarded as inappropriate because the data might not behave properly.
Machine learning, on the other hand, relies heavily on strong computational power. It thrives on high-dimensional data sets with potentially thousands of variables. There is not much need for assumptions about the behavior of the data. The whole point of machine learning is to predict, to accurately determine the outcome of something that is not represented in the data. In contrast, the whole point of classical modeling is to accurately understand what is already occurring within the data.
In the marketing/advertising space, classical approaches have historically reigned supreme. Not because they are better, but simply because they have been more widely available. However, technology (computer hardware, specifically) has improved, and word on the potential power of machine learning has spread. It is only a matter of time before traditional marketing mix models, copy testing, and media serving are all improved upon with machine learning.
Machine learning isn’t scary or even risky. Just like classical modeling, it requires specialized knowledge of math and programming. As time passes, machine learning will find more applications in our everyday lives. The approaches and techniques used by machine learning modelers are evolving seemingly every week. But one discipline is certainly here to stay: neural networks.
It’s important to peel back the math and jargon to understand, at a base level, what a neural network really is. If you have ever done any googling on artificial intelligence, you’ve probably seen a diagram like the one below.
This is a graphical representation of a neural network. Networks really only contain two components: nodes and pathways. Nodes are like checkpoints for data. Within them, the data are being transformed with any number of mathematical functions. The pathways represent the direction of the flow of the data from one node to the next. In our diagram, data flows from left to right.
We can simplify our understanding of a neural network even further. Generally, networks are thought of as being comprised of layers. In our diagram, you’ll notice that the nodes come in three colors. The gray, red, and white nodes represent the input, the hidden, and the output layers of the network, respectively. Neural networks come in a variety of different architectures, each of which is specifically designed to achieve a certain outcome.
The input and output layers are self-explanatory: Data begins its journey through the network in the input layer and ends its journey in the output layer. The output layer is usually a mathematical function that predicts something based on what the network has seen in the data. The hidden layers are where things can get complicated. Depending on the task that the network is designed to complete, you can have various different types of hidden layers as well as varying numbers of hidden layers. We consider a network to be “deep” if it has a large number of hidden layers. This is where the term “deep learning” comes from.
A large portion of advertising is having a complete understanding of what looks good. What colors play well together? What elements of an image or TV spot are appealing to consumers? In the past, we’ve relied on intuition, small sample size surveys, or trying to link advertising spend to company sales. Today, we can give a computer some knowledge, and it can help us make better decisions on our big ideas.
Neural networks can be used to accomplish many tasks. One of the more interesting overlaps between something creative like advertising and something technical like math is the arena of sight. Creatives – I imagine – are mainly concerned with how something appears. How the different elements of an image appeal to different types of viewers. Creatives are not the only ones fascinated by visuals. In fact, at The Richards Group today, strategists, mathematicians, and developers are working on building deep learning capabilities. In fact, in mid-2018, The Richards Group and its internal analytics practice, Quadratic, unleashed the agency’s first-ever neural network that can “see” images. We are using these models to better understand how elements of visual advertising tie to how different types of consumers perceive a brand and ultimately convert with that brand. We fully expect this breakthrough to give us all a fresh perspective on creative. Before we dive into the specifics of the project, we should first understand how a computer “sees.”
Computer vision and automatic content recognition are terms that vendors and journalists are using frequently these days. These are all synonymous with a single thing: convolutional neural networks (CNNs). As mentioned earlier, there are a handful of different types of hidden layers that can be combined to create a neural network architecture. CNNs gain their namesake from a specific type of hidden layer: a convolutional layer. In essence, this layer lets the network scan over an image in the same way that you might look for Waldo in a “Where’s Waldo?” puzzle: Starting at the top-left corner, moving your eyes to the right until you hit the end of the page, moving down a few inches, and starting the scanning process all over again.
CNNs are a powerful tool that allows computers to understand the context of images, videos, and much more. Your very first thought might be: “How can computers see?” Valid question. Computers don’t have eyes. They have something better: hardware. A computer’s hardware is specifically designed to crunch a lot of numbers at very high speeds. So all we have to do is make an image into numbers. This is much easier than you might think.
At their smallest level, images are just a dense grouping of colored pixels. Each pixel is simply an ordered set of three numbers, one for red, one for green, and one for blue. While we may look at an image of our friend’s recent beach getaway on Instagram and truly appreciate the beauty of the sunset, a computer will truly appreciate the beauty of pixel number 1,344, which looks like this: (255, 98, 132).
So now that we have numbers, we can perform math. With math, we can recognize, classify, and re-create patterns. Behind all of the fancy visuals and sales pitches from vendors and the jargon-filled articles, computer vision is just math, quite a bit of programming, and a healthy chunk of time. But that’s all AI is, anyway.
This all sounds straightforward, but how do we make it a reality? The answer here, for us at The Richards Group, is TensorFlow. TensorFlow is an open-source programming library developed by Google Brain, Google’s AI research department. With a solid amount of coding knowledge and a strong mathematics background, TensorFlow can be leveraged to develop all sorts of powerful machine learning models. But to get this to work, we need images. A lot of images. Thankfully, being an agency that is tuned in to the goings-on of social media, we knew exactly where to look: Instagram. Once we had our images, we needed to “clean” them. This really means bucketing our images into the groups of things we want to identify. If we wanted to identify, say, pizza, we would go through our massive selection of images and put all of the images with pizza into one place.
To accomplish this, a small team at Quadratic has cleaned and organized a data set of over 160,000 images, written over 3,000 lines of code with TensorFlow, and trained a model for over 200 hours and climbing as of October 5, 2018. Our CNN for our visual advertising research project consists of over 450 layers with over a dozen different types of nodes. The insights that this model is providing have already proved to be useful. We have verified some theories that strategists have had and rejected others. We have used the model to identify gaps in our clients’ competitors’ marketing strategies that have allowed The Richards Group to gain a strategic edge. The best part about this? We can collect more images. We can test more hypotheses. We can continue to make our models smarter.
That’s just one example of the ways that we are applying machine learning at The Richards Group and Quadratic today. We employ several techniques with interesting names like “gradient boosted trees,” “support vector machines,” and “long short-term memory networks.” We’re using machine learning to help our clients understand several things. We can learn how brands interact across geographies to identify areas ripe for growth, gain insight into the effects of natural disasters on donation revenue, choose better control and test markets for new creative, and much more.
Machine learning and artificial intelligence are an inevitable part of all of our futures – in and out of the workplace. Whether our SmartFridge is placing a new order for milk, we’re interacting with a customer service AI, or a convolutional neural network is assisting in the creative design of advertisements, we can harness the power of machine learning to be more productive and smarter than we are today. We should embrace this global shift and expend our energy determining the most tactical and intelligent ways to apply this advancement in technology. We should capitalize on this limitless opportunity to unite the minds of right-brainers and left-brainers.