Run A/B Tests directly in Shogun and start optimizing your storefront today.
Launching a test
Ready to start optimizing your pages? Great. Here is how you can launch your test with Shogun in just a few minutes.
Enable A/B Testing
Navigate to the Page you want to test in Shogun's Visual Editor. You will see a new button where you can open up the AB testing panel:
Here is where you can add a name for your A/B test, configure its objective, and add page variants.
Build your Page Variants
Now it's time to make the changes between each page variant that you want to test.
Once you have enabled A/B testing on your page, you will see a new tab at the top of the editor for each page variant. Use these tabs to quickly jump between your page variants within Shogun's Visual Editor.
When editing a Page Variant, you can make any change to your page's layout that you wish. You can test a full page redesign, or a simple text change.
Note: Autosave currently is not compatible when A/B Testing is enabled, so please Save your changes manually.
Define your Traffic
Shogun supports the ability to control the traffic distribution so that you can test bold ideas safely and with control.
Simply toggle off the "Split traffic evenly between variants" option and enter define much traffic you want each variant to receive.
Publish Test
Once you have made your desired changes to your page variants, connected Google Analytics, and set up your GA4 Audiences, you are ready to launch your test.
Simply click Publish Test and your test will go live.
On the Dashboard, add a Page Type filter to quickly find all your pages that have a test in progress.
Updating your A/B Test
If you need to make a modification to your page while you have a test in progress, Shogun will allow this. However, please be mindful that any update to a test can impact the validity of your test results.
Analyzing your Test
Viewing A/B Test Analytics in Shogun
There are multiple spots within Shogun where you can review the results of your A/B tests.
Analytics on the Shogun Dashboard
Shogun will automatically track the performance of your A/B tests and display the results in your dashboard. When clicking the "A/B tests" menu item in the left sidebar of your Shogun dashboard, you'll see an overview of all A/B tests. Click on the "View Analytics" button next to any test to view detailed results for each page.
The results of A/B tests can also be accessed through the "Page Analytics" section of your Shogun dashboard.
Viewing Analytics in the Shogun Visual Editor
When viewing an active A/B test in the visual editor, click the A/B test icon to open the test's settings.
Once the A/B test menu is open, click on the "Publish & Analyze" tab. The bottom of this view will contain an overview of the test's performance since publishing. Click on the "View detailed analytics" button to see the full metrics on how each page variant has been performing over time.
Stop your Test
When you are ready to complete your test, open the A/B test menu in the Shogun editor. Go to the "Publish and Analyze" tab and click the "End test now" button.
Then, you will be presented with the following model, from which you will choose the variant you'd like to proceed with.
FAQ
How does Shogun calculate statistical significance?
You can find the details here
Why am I seeing an old version of my live page?
When you visit a live page with a Shogun A/B Test running, Shogun updates your browser's local storage so that your shoppers have a consistent experience when visiting your store. To see a random version of your page, add ?cache=false
at the end of your live page URL and you will be served a random page variant.
β
How many MTUs (Monthly Tracked Users) does my plan support?
Great question!
You can see at the top of the dashboard how many MTUs you have at any time.
What happens when I hit my MTU limit?
MTUs (Monthly Tracked Users) are only recorded when customers visit pages that have active A/B tests or Personalised Experiences. It is important to note that each individual user is counted as 1 MTU no matter how many times they visit the page.
When you run out of MTUs, your pages will remain published, but all users will be directed to the Default Variant of the page.
β
This behavior will continue until the beginning of your next billing cycle when your MTUs are rolled back.