The evolution of the Internet has made many things possible for the enterprise that just a few decades ago would have seemed unfathomable. Retailers are able to understand their customers in more depth than ever before. The Internet of Things gives companies real-time feedback on how their products are performing.
The Internet has also helped push down decision making at all levels of the business. What do I mean by that? Well, A/B testing is in more widespread use now than ever before.
Online, marketers are doing A/B tests at various levels. While A/B testing has been used for years, it has gone through tremendous evolution over time. It started as a way to test user interface methods before advancing as a way to test more complicated offers and campaigns targeted at specific groups of consumers.
Now a whole new generation of testing is arising from companies like Split that are offering the ability to run deep, full-stack experimentation. This could radically reshape the way experimentation is done in enterprises of all sizes.
Full-stack experimentation is testing at every level of the data and analytics stack. It enables developers and engineers to run the type of testing that until now has mostly been done by a company’s marketing team.
Because of the resources and staff development skills required, only the behemoths of the online world have been able to conduct full-stack experimentation up until now. That’s why Etsy, Uber, Google, FaceBook, and others have built custom stacks to perform this type of experimentation. Smaller companies could only look on with envy at this type of capacity. Now that’s changing.
As content marketers, what is always interesting to us at Evolved Media is how can we take great products, gain an understanding of their capabilities, and then tell the story of what those products can do for businesses in a compelling way. Our goal is to communicate clearly what a product does and how it creates success for the customer. We want to provide a greater level of understanding about the product itself than consumers would normally get from #content marketing. That’s what differentiates content marketing from a #product marketing perspective.
How does that apply to A/B testing and full-stack experimentation? As I looked more deeply at the world of A/B testing, I’ve become convinced that product marketers should be working with people who are doing A/B tests to understand what’s being learned on both positive and negative use cases. As I mentioned earlier, A/B testing has often been geared towards marketing offers and marketing performance. But as full-stack experimentation gains wider adoption, companies of all sizes and in all industries will be using more feature flagging and collecting more data. This should be a goldmine for product marketers.
A product marketer should be able to dig into this data and generate ideas about what has been successful, what has not, and what types of fully formed features are most appealing to highly specific types of audiences. Full-stack experimentation allows testing of much larger changes to a product using feature flagging, which turns on and off huge blocks of code inside an application. Product marketers can use this information to create better campaigns and content, but there can also be a positive feedback loop in which they inform and work with engineers and developers to create more market-ready products.
When you do A/B tests and find A is better than B, C, D or E, that’s great information. But with full-stack experimentation, after you analyze the data, you get more granular analysis. You can break out how the product is being used and received by more minute subpopulations. Additionally, you can begin to understand which product features are the most successful.
One of the biggest boons for product marketers is the learning that take place from failed experimentation. When experimentation is done at scale, it turns out that most experiments don’t work. And when they don’t work, you start to get a feeling for what went wrong. This can be a powerful tool to steer away from product marketing messages and themes that are related to the failed experiments. Of course, using learning from what does work is vital too, but something that would be done more naturally.
As a product marketer, I’m excited about the potential full-stack experimentation holds for product marketing going forward. We believe fully in this possibility, but as of yet, we haven’t found a company that is truly incorporating full-stack experimentation or A/B testing into their product marketing in a robust way. If you work at such a company, or know of one, we’d love to talk to you, as we think that soon, product marketing will be moving way beyond the traditional world of A/B testing.
Become a product marketing master
