A/B test types
Airship provides multiple A/B testing options for various metrics and channels.
Types of A/B tests available in Airship:
| A/B test type | Metric | Channels | Description |
|---|---|---|---|
| Message | Engagement |
| Create variants of message content by duplicating the initial variant or from scratch. Each variant returns analytic data to help you determine the most effective way to engage your audience. You can retain a control group or send to 100% of your selected audience.
|
| Sequence | Conversion or engagement |
| Create a variant for any message in a SequenceA series of messages that is initiated by a trigger. Airship sends messages in the series based on your timing settings, and you can also set conditions that determine its audience and continuation. Sequences can be connected to each other and to other messaging components to create continuous user experiences in a Journey.. The variant is a duplicate of the original message that you can then edit, changing its content, delivery settings, or Channel CoordinationA group of strategies for targeting users where they are most likely to receive your message. settings. Audience allocation is set to 50% for each variant by default, but you can change the percentages. After starting the test, you will wait till the Confidence level meets or exceeds 95% and then select the winning message. The Sequence is then republished with the winning message. Audience members who receive the variant message are randomly selected on entry to the Sequence. Related events and conversions are recorded for both audiences, providing data you can use to evaluate Sequence performance based on your selected metric. You can run Sequence A/B tests and control groups concurrently.
|
| Scene |
| App (Scene) | Create variations of SceneA single or multi-screen in-app experience cached on users’ devices and displayed when users meet certain conditions in your app or website, such as viewing a particular screen or when a Custom Event occurs. They can be presented in fullscreen, modal, or embedded format using the default swipe/click mode or as a Story. Scenes can also contain survey questions. content by duplicating an existing Scene or creating screens from scratch. You can make a single change, such as changing a button label in a screen, or provide entirely different content. Audience members are randomly selected and split equally to receive your control Scene (Variant A) and your variant Scene (Variant B) for the targeted audience. Related events and conversions are recorded for both audiences, providing data you can use to evaluate Scene performance based on your selected metric.
|
| Feature Flag | GoalsSelected events that generate a set of performance reports. You can also use them for measurement in Holdout Experiments and Feature Flag A/B tests. | App or Web content | Compare audience behaviors when a feature is hidden or present, or experiment with distinct feature experiences, such as new home screen designs, by setting different property values for each variant. Reports provide detailed data for evaluating engagement and the overall success of a feature based on your Goals.
|
Categories