5 Steps to Going Beyond the AB Test

Going beyond the AB test is something that all conversion rate optimisers should do – CRO is not all about running successful but discrete tests, it’s really about creating experiences that drive conversion improvements in the metrics you care about.

And sometimes, driving an uplift in that metric is not as simple as running one page design against another; sometimes, it requires going beyond the AB test. So if you have a troublesome metric like this, consider the following 5 steps:


1. Think strategically

This is the essential place to start – if the metric you are considering is not a strategic priority for your business, it’s very unlikely that you’re going to have the scope to run multiple tests to address it. So think about what is going to get the buy-in you need, and drive results that will get you noticed.


2. Think sequentially

If this metric is as problematic as you think, it’s not going to be solved in one test – work back from the main goal and write down the issues that emerge at each step of the process. This is how you will start to form a plan because you should now be aware of the root cause of the issue. This should also enable you to plan the tests after the initial test, with your expected results informing the next stage of the project plan.


3. Think iteratively

Now you have your project outline, you’re ready to launch your first test. But be prepared for it not to deliver what you need and have iterations in mind – as the first step of your journey, it’ll be vital to the success of the overall project to get this one right. Be it segment identification or cookie collection or any other success metric, this first test has to go well, otherwise your project will be built on very unsteady ground – so be prepared to iterate if it doesn’t work straight away.


4. Think analytically

One of the biggest concerns for any manager when a report talks about a “strategic project” is that they aren’t going to see any output for a long time! So remember to deliver results to them for each stage of the project, ensuring that you explain why that test was essential to the strategic goal and which building block it provided for you to stack the next test on top of. This will keep your manager on-side and prevent them from pulling the plug on a project that could revolutionise your business.


5. Think tangentially 

You started this project with one goal in mind, and you built your test plans around that. And whilst it still needs to remain your focus for all your efforts to have been worthwhile, don’t forget to record any seemingly unrelated results along the way. Eventually your current project will have run its course and having results that can inform new projects too will only strengthen your case for going beyond the AB test at another point!

Leave a Reply

Your email address will not be published. Required fields are marked *