All Collections
Advanced
Optimizing your pages with A/B Testing
Optimizing your pages with A/B Testing

Learn more about how you can run multivariate tests and view analytics within Shogun

Updated yesterday

Run A/B Tests directly in Shogun and start optimizing your storefront today.

Launching a test

Ready to start optimizing your pages? Great. Here is how you can launch your test with Shogun in just a few minutes.

Enable A/B Testing

Navigate to the Page you want to test in Shogun's Visual Editor. You will see a new button where you can open up the AB testing panel:

Here is where you can add a name for your A/B test, configure its objective, and add page variants.

Build your Page Variants

Now it's time to make the changes between each page variant that you want to test.

Once you have enabled A/B testing on your page, you will see a new tab at the top of the editor for each page variant. Use these tabs to quickly jump between your page variants within Shogun's Visual Editor.

When editing a Page Variant, you can make any change to your page's layout that you wish. You can test a full page redesign, or a simple text change.

Note: Autosave currently is not compatible when A/B Testing is enabled, so please Save your changes manually.

Define your Traffic

Shogun supports the ability to control the traffic distribution so that you can test bold ideas safely and with control.

Simply toggle off the "Split traffic evenly between variants" option and enter define much traffic you want each variant to receive.

Publish Test

Once you have made your desired changes to your page variants, connected Google Analytics, and set up your GA4 Audiences, you are ready to launch your test.

Simply click Publish Test and your test will go live.

On the Dashboard, add a Page Type filter to quickly find all your pages that have a test in progress.

Updating your A/B Test

If you need to make a modification to your page while you have a test in progress, Shogun will allow this. However, please be mindful that any update to a test can impact the validity of your test results.

Analyzing your Test

Viewing A/B Test Analytics in Shogun

There are multiple spots within Shogun where you can review the results of your A/B tests.

Analytics on the Shogun Dashboard

Shogun will automatically track the performance of your A/B tests and display the results in your dashboard. When clicking the "A/B tests" menu item in the left sidebar of your Shogun dashboard, you'll see an overview of all A/B tests. Click on the "View Analytics" button next to any test to view detailed results for each page.

The results of A/B tests can also be accessed through the "Page Analytics" section of your Shogun dashboard.

Viewing Analytics in the Shogun Visual Editor

When viewing an active A/B test in the visual editor, click the A/B test icon to open the test's settings.

Screenshot of the A/B testing button in Shogun

Once the A/B test menu is open, click on the "Publish & Analyze" tab. The bottom of this view will contain an overview of the test's performance since publishing. Click on the "View detailed analytics" button to see the full metrics on how each page variant has been performing over time.

A/B Test Results in Google Analytics

You can also view the results of your A/B test inside Google Analytics by connecting your Shogun account to GA4. To do this, go to the Settings section of your Shogun dashboard then click the "Enable" button next to the "Google Analytics Oauth" integration option.

Screenshot of the integration settings in Shogun.

This will open a new tab where you'll be prompted to authorize the connection between Shogun and Google Analytics.

After connecting Google Analytics, Shogun will automatically create GA4 Audiences and configure event tags so that you can analyze the performance of your tests in Google Analytics.

Please note: Google imposes is a limit of 100 Audiences for each GA4 property. If you have reached the limit, please archive Audiences you no longer need and then publish your test.

Seeing an error when authenticating? Please review the following requirements:

  1. Connect Google Analytics to your ecommerce platform

  2. Ensure your storefront is not password protected

  3. Ensure your Google account has adequate permissions in your Google Analytics property

    1. Administrator or Editor permissions are required in order to complete authentication.

      Screenshot of user permissions in Google Analytics

Ensuring GA4 is tracking properly

After you publish your first test, watch the video below to ensure analytics have been connected successfully and how to add GA4 Audiences to your Reports to analyze your test.

Need help building a Report in GA4? You can learn more about building reports with custom Audiences here.

Stop your Test

When you are ready to complete your test, open the A/B test menu in the Shogun editor. Go to the "Publish and Analyze" tab and click the "End test now" button.

The button to end an A/B test in Shogun

Then, you will be presented with the following modal where you need to pick the variant you want to move forward with. Don't worry, your other variants will remain available if you'd like to access them in the future.

FAQ

How does Shogun calculate statistical significance?

You can find the details here

Why am I not seeing any data in GA4?

Event data takes GA4 24 - 48 hours to become available in your reports. While you wait for GA4 to process your event data, you can view the Realtime view to see your events coming in.

Why am I seeing an old version of my live page?

When you visit a live page with a Shogun A/B Test running, Shogun updates your browser's localstorage so that your shoppers have a consistent experience when visiting your store. To see a random version of your page, add ?cache=false at the end of your live page URL and you will be served a random page variant.

Did this answer your question?