A/B Test Composer Tutorial

Use A/B Tests to experiment with different variations, or variants, of a message for a given audience, and provide engagement reporting data for each variant.

See the Message Reference for allowed message types and combinations. See also: Message and Composer Types and Composer Overview.

 Tip

You can configure the default appearance of your In-App Messages in Settings » Configuration » In-App Message Design. This includes button text and message color, screen position, and more.

  1. Click Create, and select A/B Test.

  2. (Optional) Add a name and/or flag as a test.
    1. Click in the header.
    2. Enter the name.
    3. Enable Test.
    4. Click outside the box to close it.

    See also: Name a Message and Flag a Message as a Test.

  3. Define your A/B test Audience. This is the entire set of users for the A/B test, which includes the control group (users who receive no message) and users who will receive different messages, according to the variants that you define.

    1. Set the ChannelsA communication medium supported by the Airship service. Supported channels include app, web, email, SMS, and Open Channels. Within some channels there may be specific platforms with individual characteristics. Example platforms include Chrome for the web channel and Android for the mobile app channel. that you want to send messages to. Only channels available for A/B testing are listed. Features and options available in the composer are reduced to those common to the channels you select.

    2. Select the group of users you want to send your message to.
      • All Users: Sends the message to your entire audience.
      • Target Specific Users: Create a recipient group based on segmentation data. See: Target Specific Users.
      • Test Users: Predefined recipient groups. After choosing Test Users, select from the Test Groups dropdown menu that appears. See: Test Groups.
    3. Click Variants in the header to move on.

  4. Set up your message Variants.

    1. Enter a descriptive Test Name if you didn't enter a name in the Audience step.

    2. Select the Number of Variants for your test, up to 26. You can add or delete variants in later steps.

    3. Use the slider to adjust the percentages of your target audience vs. control group. By default we send your test to 80% of your audience, keeping a control group of 20%

    4. Click Content in the header to move on.

  5. Add your message Content. The left side of the screen has a tab for each variant. Start with variant A. Message content options may change based on the channels and message types in your message.

    1. Choose a message type, then click Add Content. The message type selection screen only appears if you selected app channels.

    2. Enter a Variant Name.

    3. Enter your message text.For app channels, when you combine a push notification and in-app message, the in-app message's alert text is the same as the push notification text by default. To use different messages, select Write Alternative for In-App Message Text and enter your text.

    4. Select a message action. Available actions depend on the message type(s) you selected.

      Adaptive LinkAdaptive Link opens a mobile wallet pass. Select an Adaptive LinkA mobile wallet pass link that supports templates for both Google Pay and Apple Wallet. When a user taps the link, Airship determines the user's platform and generates the right pass for the user. from the list.

      • Adaptive links must be configured before they will appear in the actions list. See: Adaptive Links Tutorial.
      • Only adaptive links created in the dashboard will appear in the actions list.
      Deep LinkDeep Link opens a screen in your app or website. Select a deep link from the list.

      HomeHome opens your app's home screen. For web push notifications it opens your Default Action URL. You can override the default URL by selecting the Web Page action and entering a different URL.
      Landing Page

      Landing Page opens a rich pageA landing page or Message Center message in your app that can include HTML, video, etc. that appears until it is closed. Follow the steps in Create Rich Pages. See also: Rich Page Reference.

       Note

      The default behavior in the iOS SDK is to not display the landing page when the corresponding push notification is received while the app is in the foreground state. The logic behind this behavior is that you may not wish to interrupt or distract a user who is currently occupied with the app, only to ask them to visit a different part of the app.

      If you need to display a landing page while the app is in the foreground state, replace the registry predicate associated with that action with one that allows foreground execution.

      In the Airship SDK, this relative state between what the app is doing now and how the action was invoked is known as a situation, or more properly, UASituation. Other examples of situations besides foreground state include background push and app launched from push notification.

      Please see the UA Action Arguments Class Reference for more detail on the methods involved with this display behavior.

      Message CenterMessage Center opens a rich pageA landing page or Message Center message in your app that can include HTML, video, etc. that is saved in the user's Message CenterA place in your app where you can display persistent rich messages, including HTML, video, etc. Similar to email, Message Center represents both the medium (the in-app inbox) and the message type (the messages you send to a message center). inbox. Follow the steps in Create Rich Pages. See also: Rich Page Reference.
      ShareShare prompts the user to share your message with apps, social media accounts, and other services. Enter the text you want to accompany the share, including any promotional information, shortened links, hashtags, etc.
      Web PageWeb Page opens a web page or any valid-device level URL such as App Store or app protocol links. The web page opens in the device's default browser. Enter a URL.

    5. (Optional) Set and/or remove tags when the user interacts with your message.
      1. Click Set a tag.
      2. Select Add or Remove, then search for tags that exist in the system, or create a new tag.
      3. (Optional) Click Set another tag and repeat the previous step.

       Tip

      Setting one or more tags when a user interacts with a message can help you track user interactions for follow-on retargeting campaigns. For example, if you set a tag "responded-campaign1", you can target users bearing the "responded-campaign1" tag with another message at a later date, knowing that they are active users. Alternatively, you can re-engage users with this tag using an automated message if they are inactive for a period of time.

    6. Set Optional Features.

    7. Configure additional variants. You can add or delete variants by clicking and .

      1. Select a variant using the lettered tabs.

      2. Copy content from an existing variant, or start with a blank message. If copying content, select a variant, then click Continue.

      3. Fill out the message variant including text, action, and options — like you did for variant A.

    8. Click Delivery in the header to move on.

  6. Set Delivery options for your A/B test.

    1. Select a delivery type.

      Send nowSend the messages immediately after review.
      ScheduleChoose an exact time of day to send the message. Enter a date in YYYY-MM-DD format and select the time and time zone.

    2. Set Optional Features.

    3. Click Review in the header to move on.

  7. Review your A/B test.

    1. Select a variant in the Content section or above the preview.

    2. Review the device preview and message summary.Click the arrows to page through the various previews. The channel and display type dynamically update in the dropdown menu above. You can also select a preview directly from the dropdown menu.If you would like to make changes, click the associated step in the header, make your changes, then return to Review.
    3. Click Send Message or Schedule Message.