A/B Testing for Scenes

Scenes are multi-screen experiences that are cached on users’ devices and displayed when your users meet certain conditions within your app, such as viewing a particular screen or opening the app a certain number of times.

Today’s release gives you A/B testing capabilities, as well as updates to the Content step when creating scenes.

Scene A/B testing

You can make a single change, e.g., changing a button label in a screen, or provide entirely different content for a scene. You can create the variant from scratch or by editing a duplicate of the scene.

Audience members are randomly selected and split equally to receive your control scene (Variant A) and your variant scene (Variant B) for the targeted audience. Related events and conversions are recorded for both audiences, providing data you can use to evaluate scene performance based on your selected metric for the test:

• Scene completion — The user viewed all screens in the scene.
• Push Opt-in — The user clicked a button configured with the Push Opt-in action.
• Adaptive Link — The user followed an adaptive link in the scene.
• App Rating — The user clicked a button configured with the App Rating action.
• Deep Link — The user followed a deep link in the scene.
• Preference Center — The user opened the preference center in your app.
• App Settings — The user opened their device’s settings page for your app.
• Share — The user used the Share action in your app.
• Web Page — The user opened the web page configured for the Web Page action in your app.

After you start the A/B test, you can compare the performance of the original message and variant in the scene’s Content step or in its report to determine which (or if either) message is having the expected impact.

When you select a winning scene, the scene is republished with the winner and the A/B test ends.

A/B testing for scenes requires minimum SDKs: iOS 16.6 and Android 16.4.

Updates to scene content

We removed layouts to give you more design flexibility. When creating a scene, instead of selecting a layout, add individual elements and drag them into your preferred order:

• Text — Select Header, Body, or Footer and enter text. The footer also functions as a button, so you will also configure an ActionA configurable behavior that occurs when a user interacts with your message, e.g., opening a web page.: Home, Deep Link, Adaptive Link, or Web Page.

• List — A bulleted list was previously available in the Bullet list layout only. Now you can select the List element in any screen. Provide media that serves as the bullet and enter text for the list item.

• Buttons — Previously, buttons were pinned to the bottom of the screen outside of the scrollable content. Now this can be enabled as an option.

• Media — Including media was previously available in various layouts. Now you can select the Media element in any screen. Creating a screen with media only was previously available in the Media only layout. Now background media is configured in a screen’s Design section.

These changes are available for new and existing scenes.

Documentation

• About Scenes
• Create a scene
• Scene A/B Tests