Go to Messages » Messages Overview.
Click for an A/B test. The report contains these main elements:
- Header: Displays the test name, sent date, time, and time zone.
- Performance: Contains statistical data for each variant and the control group. Click a variant name to view its message report.
- Message Detail: Contains the same information shown in the Review step of the A/B Test composer and the Message Detail section of a message report. Click the arrows to page through the various previews. The platform and display type dynamically update in the dropdown menu above. You can also select a preview or variant directly from the dropdown menus.
(Optional) Click Download CSV and select Performance Data or Variant Event Data.
Engagement data is sent to Airship as soon as it becomes available. Often data is delayed because of connectivity issues with a user’s carrier, wifi, power, etc. You should wait at least 12 to 24 hours before acting on this data to allow for potential lags.
The following data are included in the A/B Test Performance Data export:
|Project Name||Your project name.|
|Key||Unique authentication key for your app.|
|Channel Identifier||Unique identifier that refers to the entire push send. The channel identifier will be the same for each variant within an A/B Test, with N/A for users in the control group who do not receive a notification.|
|Sent Time (UTC)||UTC timestamp.|
|Test Name||User-friendly name for your A/B Test.|
|Variant||The letter corresponding to the message variant. A, B, C, etc.|
|Audience||Description of the selected audience for the A/B Test. One of All Users, Target Specific Users, or Test Users.|
|Notification Alert||Alert text for the given variant.|
|Notifications Sent/Control Sample Size||Audience send counts per variant/control group size.|
|Indirect Opens||Opens that occur within 12 hours of the notification, regardless of attribution.|
|Direct Opens||Opens that are attributed directly to interacting with the notification.|
|Influenced Opens||See: Reports: Influenced Opens.|
Variant Event Data
If you are using Custom Events, you can download a report of event conversions and associated values, broken out by variant or control group.
The following data are included in the A/B Test Variant Event Data export:
- Project Name
- Channel Identifier
- Test Name
- Event Name
- Notification Attribution