Create an A/B Test

Use A/B Tests to experiment with different variations, or variants, of a message for a given audience, and provide engagement reporting data for each variant.

To get started, click   and select A/B Test. After completing a step, click the next one in the header to move on. Click   if you want to name the message or flag it as a test.

Audience

Enable the channels you want to send the message to, then choose a group of users. This is the entire audience for the A/B test, which includes the control group (users who receive no message) and users who will receive different messages, according to the variants that you define. See: Selecting your audience.

Variants

Enter a descriptive Test name, select the Number of variants (you can add or delete variants in the Content step), and use the slider to adjust the percentages of your target audience vs. control group. By default we send your test to 80% of your audience, keeping a control group of 20%.

Content

A lettered tab for each variant is on the left side of the screen. You can click   and   to add and remove variants.

Enter a name for the first variant, then configure the message content per enabled channel. See: Creating content and Optional Features.

Click the next lettered tab and choose whether to copy content from an existing variant, or start with a blank message. If copying content, select a variant and click Continue. Enter a name for this variant, and create content for each channel. Repeat for each variant.

Delivery

Set the delivery timing and options.

  • Send now — Send the messages immediately after review.
  • Schedule —Choose an exact time of day to send the message. Enter a date in YYYY-MM-DD format and select the time and time zone.

If your message includes External Data FeedsA connection to an external API. When you send a message, Airship uses a response from that API to personalize messages. , you must configure each feed listed in External data feed options.

  • Failure behavior — Determine how your message is handled if the feed fails. Select Abort sending the message or Send message without this data.
  • Default value for [var] — This displays the default value for each message-level variable used in your data feed’s request URL. You set the default value when you Add a data feed in your project’s settings. You can change the value to override the default for this message only.

Review

Review the device preview and message summary. You can select a variant in the Content section or above the preview. Click the arrows to page through the various previews. The channel and display type dynamically update in the dropdown menu above. You can also select a preview directly from the dropdown menu.

If you want to make changes, click the associated step in the header, make your changes, then return to Review.

Click Send Message or Schedule Message.