To level set, let’s begin with something all of us are familiar with… A/B Testing.
When you conduct an AB test, you essentially take one component of a website at a time and test it in a live environment. It’s also referred to as “split testing’ because the web traffic is divided into options tested. Similarly, when you conduct a multivariate test (MVT), you are testing more than one component of a website in a live environment.
To understand how artificial intelligence (AI) fits into a testing environment, it’s helpful to break down AB / MVT into two key components:
1. Objective function (aka success): This is essentially your goal. Usually you would have your goal set up as an e-mail form submit, a registration for one your offers or a purchase transaction.
2. Offer function (aka decision): This is what you want to present to the end user (Experience / Offer A vs. Experience or Offer B) for A/B Testing or a combination of levels if you are performing an MVT.
In the simple example above, the objective function is, let’s say a purchase, while the offer function is to show red vs. green content.
Now what happens if you want to scale? Instead of performing an A/B test in one page, what if you have multiple pages where you are performing your tests with the same goal of purchase as exhibited below?
In the illustration above, we are adding collision (or complexity) to our A/B testing problem. This is because users can access multiple pages between the tests we are running on the website.
The above illustration will hopefully help drive the point of how a seemingly simple test exercise can create a complex attribution problem. This problem compounds if you are performing your tests at various touch points across lines of business or across channels. This type of attribution problem is where AI fits well in the context of testing.
Arguably, you could accomplish the same thing by running iterative tests. Specifically, you could run Test 1 and then potentially run Test 2 (determine the best case / local maximum for each test). This is obviously time consuming. In addition, this doesn’t consider other interaction effects. One could also combine Tests 1 and 2 into a multivariate test but the risk with that is you may not have all users going through each of the offers. Thus, you wouldn’t be accounting for the behaviors of the end user in this dynamic situation.
The most efficient way to solve such a problem lies in machine learning. Here’s an analogy to support this point:
Say you have a dog that is not well trained and barks every time guests come into your apartment. Now, every time the dog barks at the guest you can reduce the amount of treat (reward) you give to the dog and every time the dog doesn’t bark at the guest you double the amount of treat. What will the dog learn eventually? That barking at guests in the house does not result in getting treats.
The same analogy can be applied to the above problem via reinforcement learning (RL). In the above example, the web page is the environment, and the conversion action (buying something) is the reward and feedback given to the RL system, but this feedback is vague. It doesn’t mean anything to the machine, however eventually, with enough observations, the algorithm will figure out the relationships between Offer and Objective function.
In the above example, we could consider solving the problem via machine learning or with an RL agent that can find the best set of actions to maximize the objective function.
Here are some specific use cases that content marketers can perform by leveraging machine learning:
- Discover the types of content / offer that maximizes objective function and replicate them.
- Map content against your buyer persona (e.g., CEO, CMO, LOB, sales decision-maker. etc.).
- Provide relevant content based on where that person is within the “journey” (research, education or ready to purchase).
Good content creation always considers audience or business interest with specific problems. When powered with an unconventional RL process, you can move into a content logistics (supply chain) process where RL can provide relevant content in multiple formats to multiple roles. As a content marketer, it’s worth your while to embrace these approaches to engage your audience in a relevant and meaningful way.
