Feature Flag A/B Testing

Use Feature Flag A/B testing to make informed, data-driven decisions about features in your app or website. Optimize user engagement and confidently roll out features that drive better business outcomes.

A Feature Flag is an experimentation tool for controlling the availability of content or functionality in your app or website. Now you can use flags to compare audience behaviors when a feature is hidden or present, or experiment with distinct feature experiences, such as new home screen designs, by setting different property values for each variant.

A/B test use cases:

  • Evaluating engagement of new designs — Create an experiment to test the effectiveness of your new home screen design with new users. Display the new design to 50% of new users and the current home screen to the other 50%, set a goal such as a purchase, and track which version of the home screen leads to more conversions. If the old design still outperforms, you can stop the experiment, and if the new one wins, you can create a new rollout from the winning variant.

  • Optimizing loyalty programs — Create an experiment to test different reward structures for your new loyalty program. Create an experiment with two variations of the program: one offering discounts on future orders and another offering free delivery credits, and set a goal to track repeat orders. Reporting data reveals a 20% increase in repeat orders for the delivery credit variant, providing the team with concrete evidence to present to leadership on which program structure performs best.

Reports provide detailed data for evaluating engagement and the overall success of a feature based on your goals. After enough data is available and time has elapsed, Airship declares the winning variant. Then you can roll out that variant to 100% of your A/B test audience.

All Airship customers have one complimentary Feature Flag per project.

Requirements

Feature Flags are available as a plan add-on. AXP Essentials and Essentials Starter plan customers can navigate to the Account Info section by clicking on the account icon () at the top right of the navigation bar in the Airship dashboard. Then select Manage Add-ons, and see the Feature Flags add-on option.

If you have an enterprise plan, contact Airship Sales. See also What plan do I have? in our Feature packages reference.

Minimum SDK versions for Feature Flag A/B testing: iOS 19.0 and Android 19.0.

Documentation

Get all the details in the docs: