Test a sequence

Use sequence testing to verify the appearance and behavior of your messages, as well as channel configuration, tag conditions, and conversion or cancellation events.

You can send a sequence to a test audience according to your specified timing, then view versions of the Performance ReportA report that compares audience behavior to a sequence’s goal. It displays performance metrics and a link to the message report for each message in the sequence. and message reports based on the test audience. You can also test individual messages in the sequence.

Your test audience

You can send a test to either a single Named UserA customer-provided identifier used for mapping multiple devices and channels to a specific individual. or to a Test GroupA preview group is an audience group you create for previewing personalized content in the dashboard. Wherever a personalization preview is available, you can select a preview group, and those users’ attributes will appear for any Handlebars references to attributes. You can enable any preview group as a test group so the users can receive test messages sent to the group. These messages appear as tests in Messages Overview. .

  • If you used tag conditions in your sequence, make sure you know which users in your test audience have which tags.
  • If you personalized messages using attributes, assign the relevant attributes to your test audience.
  • There may be a discrepancy between the number of intended recipients and the number of actual devices if members of the test audience have uninstalled.

Test a sequence

Follow these steps to test a sequence that has never been started, or started but never edited.

 Note

The option to test a sequence is not available until the sequence is configured with the minimum requirements to start it.

If you just created the sequence, you will already be on the Manage screen. Otherwise, first go to Messages » Messages Overview and click   for a sequence. Then:

  1. Click Test Run in the leftside drawer.
  2. Click Set up a test run.
  3. Enter at least one named user or test group and select from the results.
  4. Specify the amount of time you want between delivery of each message in your test sequence.
  5. Click Create test.
  6. Click Start test on the Test screen. You can repeat this step to restart the test.

Now you can evaluate the test.

Test your changes after editing a sequence

Follow these steps to test your changes after editing a started or paused sequence.

If you just edited the sequence, you will already be on the Manage screen. Otherwise, first go to Messages » Messages Overview and click   for a sequence. Then:

  1. Click Test changes next to the notice You have unpublished changes.
    1. If you have created a test for this sequence before, click Yes, create new test to confirm deleting previous test data.
  2. Enter at least one named user or test group and select from the results.
  3. Specify the amount of time you want between delivery of each message in your test sequence.
  4. Click Create test.
  5. Click Start test run. You can repeat this step to restart the test.

Now you can evaluate the test.

Test a message in a sequence

Send a test message, then verify its appearance and behavior on each channel the message is configured for. The message is sent to your selected recipients immediately, and it appears as a test in Messages OverviewA view of all your project’s messages, with options for editing their settings, content, status, and more. .

Send a test from the Review step when composing a message:

  1. Click Send Test.
  2. Enter at least one named user or Test GroupA preview group is an audience group you create for previewing personalized content in the dashboard. Wherever a personalization preview is available, you can select a preview group, and those users’ attributes will appear for any Handlebars references to attributes. You can enable any preview group as a test group so the users can receive test messages sent to the group. These messages appear as tests in Messages Overview. and select from the results.
  3. Click Send.

Send a test from the Sequence ManagerA preview of the messages in a sequence, with options for editing and testing, and for running experiments. :

  1. Go Messages » Messages Overview and click   for a sequence.
  2. Click   for the message you want to test, and select Send Test Message.
  3. Enter at least one named user or test group and select from the results.
  4. Click Send.

Evaluate a sequence test

Sequence tests are manually triggered, so the first message in the sequence is sent to every member of the test audience, then the audience is filtered by channel availability and tag conditions.

After you start a sequence test, verify:

View a sequence Test report

  1. Go to Messages » Messages Overview and click   for a sequence.

  2. Click Test Run in the leftside drawer.

  3. Click View test report. The report has these main elements:

    • Performance metrics: Statistical data for the sequence and each message. See Performance Data on this page.
    • Message reports: Click   Report to open an individual message report.

Performance data

The following data are included in the Test report:

Triggers
The trigger used to initiate the sequence (tag change, custom event, inactivity, etc.) and the total number of times it was triggered. The trigger and count are located in the banner at the top of the report.
 Note

Users may trigger the sequence more than once, depending on the sequence’s rule limitLimits that cap the number of times any member of your audience can trigger a sequence, helping prevent you from over-notifying your audience, e.g., a maximum of 1 per day. Rule limits are set per sequence and apply at the named user level. , so this count may be higher than the number of individual users who triggered the sequence.

Conversions / Reactivations
The number of users who exited the sequence by a Conversion event. If your sequence does not have a Conversion outcome, this count will not appear. For sequences using the Inactivity trigger, Conversions is instead labeled Reactivations. The conversion/reactivation count is located above   Filtered for each message.
Filters and counts
The reasons why users exited the sequence, or fell out of the funnel. Filters and their counts are listed under   Filtered, to the left of each message in the sequence. Filters include:
  • Conditions — The number of users who exited due to not meeting the message’s tag conditions, e.g., message 2 was only delivered to users with tag hotdog.
  • Rescheduled — The number of users awaiting the scheduled first message in a sequence who then performed the trigger event again, resulting in rescheduling the message delivery.
  • Other — The number of users who exited due to other reasons.
Users eligible for message
The total number of users that have met the criteria to receive the message.
Direct engagements
  • The total number of direct app opens and clicks on web notifications.
  • The percentage of direct engagements is calculated as Direct Engagements / Number of Sends * 100.