Amazon Web Services

AIRSHIP MAINTAINED INTEGRATION

This integration is maintained by Airship. Please contact Airship for support.

Airship Real-Time Data StreamingA service that delivers engagement events in real time via the Data Streaming API or an Airship partner integration. currently supports two integrations with Amazon Web Services: S3 and Kinesis. This document walks through the process of connecting to each service.

AWS Integration Requirements

Accounts
1. AWS S3 or Kinesis
2. Airship — Must include both:
  - Messaging
  - Real-Time Data Streaming

Amazon S3

S3 is a cloud storage service. Once you have integrated Real-Time Data Streaming with S3, your event stream data should begin funnelling into a bucket. From there, how you use the data is up to you, but some potential ideas are:

Output CSV files with user-level send and open information, and import these files into your CRM system.
Output your data in JSON format, and use that data along with Amazon Redshift to perform detailed analysis of your users.

Tip

Be sure to regularly audit your event stream S3 bucket. Real-Time Data Streaming outputs large amounts of data, which can lead to expensive AWS bills if not managed appropriately.

Setting up AWS S3

Log in to Amazon Web Services . Click Services and go to S3.
Create a bucket. Follow Amazon’s process for creating an S3 bucket, selecting the appropriate S3 Region for your project. For more information on S3 regions, see Amazon Regions .
Grant permissions to your newly-created bucket.
1. Go to Permissions » Access Control List.
2. Click Add Account, and use Airship’s ID as the canonical ID / account field: 7e7585ea39ccec40d8297a9038ba7f211b1c4a48994c2c702298aca8732f9f0e
3. Select List objects and Write objects. You can select additional permissions, but your bucket must have these permissions to support an Airship integration.
4. Click Save.

Configuring the AWS S3 Integration

You will need the bucket name from Setting up AWS S3.

In Airship:

Go to Settings » Project Configuration and click Manage for Real-Time Data Streaming.
Under Real-Time Data Streaming, select Amazon S3. Previously configured integrations are listed under Enabled Integrations.
Follow the onscreen instructions to configure the integration.
You can select JSON or CSV as the output format and have the option to encrypt your data using server-side encryption or export as a compressed gzip file. Tag Changes are not supported by CSV because their dynamic nature cannot be easily represented in a single row column structure.

Output Structure and Files

Your S3 bucket’s directory structure and files will be named using the following patterns:

CSV: appKey + “/” + integrationId + “/S3_CSV/” + eventType + “/” + year + “_” + month + “_” + day + “/” + year + “_” + month + “_” + day + “_” + hour + “_” + minute + “_” + second + “.csv”
For a list of headers for each supported event type, see S3 CSV Headers below.
JSON: appKey + “/” + integrationId + “/S3_JSON/” + year + “_” + month + “_” + day + “/” + year + “_” + month + “_” + day + “_” + hour + “_” + minute + “_” + second + “.json”

There will be one file generated per hour, assuming the relevant event occurred during that hour.

Note

File size maximum is 15 GB. If the volume of events for an app results in a file exceeding 15 GB, you may instead see multiple files.

Sample File

Here are a few lines from a sample CSV file for LOCATION events:

Nonet.id,event.type,event.occurred,device.platform,device.channel_id,device.named_user_id,LOCATION.latitude,LOCATION.longitude,LOCATION.foreground,LOCATION.session_id
00000000-0000-0000-0000-000000000000,LOCATION,2015-11-18T01:21:33.180Z,IOS,90823094-1234-94b2-sb39-099s9018gx55,,14.5224123,-22.1236212,false,
00000000-0000-0000-0000-000000000000,LOCATION,2015-11-18T01:21:33.171Z,IOS,90823094-1234-94b2-sb39-099s9018gx55,,14.5224123,-22.1236212,false,
00000000-0000-0000-0000-000000000000,LOCATION,2015-11-18T01:21:33.162Z,IOS,90823094-1234-94b2-sb39-099s9018gx55,,14.5224123,-22.1236212,false,
...

If you choose to output your data as JSON, the files will follow the format laid out in the Data Streaming API Reference.

Amazon Kinesis

Kinesis is AWS’s conduit for streaming data. Once you have integrated Real-Time Data Streaming with Kinesis, you will be able to load and analyze mobile events in real time.

Once you have Airship data running through a functioning Kinesis stream, you will likely want to begin analyzing and processing that data. To that end, AWS Lambda is a backend compute service that can process events in real time. Lambda responds to events that occur on other AWS products, such as a Kinesis stream.

As an example, let’s say you have built out a mapping between Named User IDs and associated email addresses, and you would like to send an email to any user that chooses to uninstall your app. Here’s a typical app uninstall event:

{
   "id" : "ff76bb85-74bc-4511-a3bf-11b6117784db",
   "type": "UNINSTALL",
   "offset": 1235,
   "occurred": "2015-05-03T02:32:12.088Z",
   "processed": "2015-05-03T12:12:43.180Z",
   "device": {"named_user_id": "named-user-id-123"}
}

You would then use Lambda to create a function associated with your Airship-Kinesis stream. The function should search for UNINSTALL type events that occurred on devices with associated Named User IDs and then send those devices an email:

def uninstall_email(event, context):
   if (event['type'] == 'UNINSTALL' and 'named_user_id' in event['device']):
      send_email(event['device']['named_user_id'])

The above pseudo-code finds events that fit the parameters and then uses the send_email function to send an email to the appropriate Named User.

There are thousands of ways to process and analyze your Airship data with Kinesis and Lambda, e.g., editing your CRM database when certain events come in, writing streams into Redshift or Dynamo DB, monitoring tag changes. The possibilities are endless.

Setting up AWS Kinesis

Log in to the AWS Console.
Create a Kinesis Stream. Go to Kinesis in the AWS console, click Create Data Stream, and follow the steps. When complete, copy the ARN; you will need the ARN when creating a policy.
Create a Policy. The following represents the minimum policy to support Airship Real-Time Data Streaming.
1. Go to Services and search for IAM — Identity and Access Management.
2. Go to Policies, and click Create Policy
3. For the Service select Kinesis
4. Assign Actions for the policy. At a minimum, you should assign DescribeStream, GetShardIterator, GenerateKeyData,GetRecords, PutRecord, and PutRecords (found under Read and Write categories, respectively).
5. For Resources provide the ARN.
6. Review and then submit the policy.
Create a User to attach the policy to. This user will represent Airship in your Kinesis instance.
1. Go to Users and click Add User.
2. Enter a User name and select Programmatic Access for the AWS Access Type, and then click Next.
3. Under Set permissions boundary, select Use a permissions boundary to control the maximum user permissions and select the policy you created in previous steps.
4. When you finish creating your user, copy the Access key ID and Secret access key — you will need these when enabling your Kinesis integration in Airship.

Configuring the AWS Kinesis Integration

You will need your Access Key, Access Secret, and Kinesis Stream Name. You can get the Access Key ID and Secret Access ID from the User you created for your integration.

In Airship:

Go to Settings » Project Configuration and click Manage for Real-Time Data Streaming.
Under Real-Time Data Streaming, select Amazon Web Services / Amazon Kinesis. Previously configured integrations are listed under Enabled Integrations.
Follow the onscreen instructions to configure the integration.

Appendix: S3 CSV Headers

You may choose the CSV format for your Amazon S3 data output. These are the headers for supported event types.

OPEN: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE, LAST_DELIVERED_PUSH_ID, LAST_DELIVERED_GROUP_ID, LAST_DELIVERED_VARIANT_ID, LAST_DELIVERED_TIME, TRIGGERING_PUSH_ID, TRIGGERING_GROUP_ID, TRIGGERING_VARIANT_ID, TRIGGERING_TIME, SESSION
CLOSE: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE, SESSION
CUSTOM: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE, LAST_DELIVERED_PUSH_ID, LAST_DELIVERED_GROUP_ID, LAST_DELIVERED_VARIANT_ID, LAST_DELIVERED_TIME, TRIGGERING_PUSH_ID, TRIGGERING_GROUP_ID, TRIGGERING_VARIANT_ID, TRIGGERING_TIME, SESSION, NAME, VALUE, TRANSACTION, INTERACTION_ID, INTERACTION_TYPE, CUSTOMER_ID
LOCATION: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE, SESSION, LATITUDE, LONGITUDE, FOREGROUND
SEND: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE, PUSH_ID, GROUP_ID, VARIANT_ID
TAG_CHANGE: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE
RICH_DELETE: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE, PUSH_ID, GROUP_ID, VARIANT_ID
RICH_DELIVERY: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE, PUSH_ID, GROUP_ID, VARIANT_ID
RICH_READ: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE, PUSH_ID, GROUP_ID, VARIANT_ID
REGION: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE, SESSION, ACTION, REGION_ID, NAME, SOURCE, SOURCE_ID
IN_APP_MESSAGE_DISPLAY: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE, PUSH_ID, GROUP_ID, VARIANT_ID, TRIGGERING_PUSH_ID, TRIGGERING_GROUP_ID, TRIGGERING_VARIANT_ID, TRIGGERING_TIME, SESSION
IN_APP_MESSAGE_EXPIRATION: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE, PUSH_ID, GROUP_ID, VARIANT_ID, TRIGGERING_PUSH_ID, TRIGGERING_GROUP_ID, TRIGGERING_VARIANT_ID, TRIGGERING_TIME, SESSION, TIME_SENT, RESOLUTION_TYPE, TIME_EXPIRED, REPLACING_PUSH_PUSH_ID, REPLACING_PUSH_GROUP_ID, REPLACING_PUSH_VARIANT_ID, REPLACING_PUSH_TIME
IN_APP_MESSAGE_RESOLUTION: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE, PUSH_ID, GROUP_ID, VARIANT_ID, TRIGGERING_PUSH_ID, TRIGGERING_GROUP_ID, TRIGGERING_VARIANT_ID, TRIGGERING_TIME, SESSION, TIME_SENT, RESOLUTION_TYPE, BUTTON_ID, BUTTON_DESCRIPTION, DURATION
CONTROL: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE, PUSH_ID, GROUP_ID, VARIANT_ID
SCREEN_VIEWED: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE, SESSION, DURATION, PREVIOUS_SCREEN, VIEWED_SCREEN
FIRST_OPEN: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE
UNINSTALL: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE
PUSH_BODY: ID, TYPE, OCCURRED, PROCESSED, OFFSET, PUSH_ID, GROUP_ID, VARIANT_ID, PAYLOAD, TRIMMED, RESOURCE
WEB_CLICK: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE, PUSH_ID, GROUP_ID, VARIANT_ID
WEB_SESSION: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE, LAST_DELIVERED_PUSH_ID, LAST_DELIVERED_GROUP_ID, LAST_DELIVERED_VARIANT_ID, LAST_DELIVERED_TIME, SESSION
IN_APP_BUTTON_TAP: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE, APP_PACKAGE_NAME, APP_VERSION DEVICE_MODEL, DEVICE_OS, CARRIER, PUSH_OPT_IN, BACKGROUND_PUSH_ENABLED, LOCATION_ENABLED, LOCALE_LANGUAGE_CODE, LOCALE_COUNTRY_CODE, LOCATION_PERMISSION, IANA_TIMEZONE, APP_DEFINED_ID, BUTTON_ID, GROUP_ID, RENDERED_LOCALE, PUSH_ID, SESSION, TIME_SENT, TRIGGERING_PUSH_ID, TRIGGERING_GROUP_ID, TRIGGERING_VARIANT_ID, TRIGGERING_TIME
IN_APP_EXPERIENCES: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE, APP_PACKAGE_NAME, APP_VERSION DEVICE_MODEL, DEVICE_OS, CARRIER, PUSH_OPT_IN, BACKGROUND_PUSH_ENABLED, LOCATION_ENABLED, LOCALE_LANGUAGE_CODE, LOCALE_COUNTRY_CODE, LOCATION_PERMISSION, IANA_TIMEZONE, APP_DEFINED_ID, EVENT_NAME, GROUP_ID, PUSH_ID, SESSION, SURVEY_TYPE, TIME_SENT, TRIGGERING_PUSH_ID, TRIGGERING_GROUP_ID, TRIGGERING_VARIANT_ID, TRIGGERING_TIME, VARIANT_ID
IN_APP_FORM_DISPLAY: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE, APP_PACKAGE_NAME, APP_VERSION DEVICE_MODEL, DEVICE_OS, CARRIER, PUSH_OPT_IN, BACKGROUND_PUSH_ENABLED, LOCATION_ENABLED, LOCALE_LANGUAGE_CODE, LOCALE_COUNTRY_CODE, LOCATION_PERMISSION, IANA_TIMEZONE, APP_DEFINED_ID, GROUP_ID, PUSH_ID, SESSION, TIME_SENT, TRIGGERING_PUSH_ID, TRIGGERING_GROUP_ID, TRIGGERING_VARIANT_ID, TRIGGERING_TIME, IN_APP_FORM_EVENT_TYPE
IN_APP_FORM_RESULT: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE, APP_PACKAGE_NAME, APP_VERSION DEVICE_MODEL, DEVICE_OS, CARRIER, PUSH_OPT_IN, BACKGROUND_PUSH_ENABLED, LOCATION_ENABLED, LOCALE_LANGUAGE_CODE, LOCALE_COUNTRY_CODE, LOCATION_PERMISSION, IANA_TIMEZONE, APP_DEFINED_ID, GROUP_ID, PUSH_ID, SESSION, TIME_SENT, TRIGGERING_PUSH_ID, TRIGGERING_GROUP_ID, TRIGGERING_VARIANT_ID, TRIGGERING_TIME, IN_APP_FORM_EVENT_TYPE
IN_APP_PAGE_SWIPE: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE, APP_PACKAGE_NAME, APP_VERSION DEVICE_MODEL, DEVICE_OS, CARRIER, PUSH_OPT_IN, BACKGROUND_PUSH_ENABLED, LOCATION_ENABLED, LOCALE_LANGUAGE_CODE, LOCALE_COUNTRY_CODE, LOCATION_PERMISSION, IANA_TIMEZONE, APP_DEFINED_ID, FROM_PAGE_INDEX, FROM_PAGE_IDENTIFIER, GROUP_ID, RENDERED_LOCALE, PAGER_IDENTIFIER, PUSH_ID, SESSION, TIME_SENT, TO_PAGE_IDENTIFIER, TO_PAGE_INDEX, TRIGGERING_PUSH_ID, TRIGGERING_GROUP_ID, TRIGGERING_VARIANT_ID, TRIGGERING_TIME
IN_APP_PAGE_VIEW: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE, APP_PACKAGE_NAME, APP_VERSION DEVICE_MODEL, DEVICE_OS, CARRIER, PUSH_OPT_IN, BACKGROUND_PUSH_ENABLED, LOCATION_ENABLED, LOCALE_LANGUAGE_CODE, LOCALE_COUNTRY_CODE, LOCATION_PERMISSION, IANA_TIMEZONE, APP_DEFINED_ID, COMPLETED, GROUP_ID, RENDERED_LOCALE, PAGE_COUNT, PAGE_IDENTIFIER, PAGE_INDEX, PAGER_IDENTIFIER, PUSH_ID, SESSION, TIME_SENT, TRIGGERING_PUSH_ID, TRIGGERING_GROUP_ID, TRIGGERING_VARIANT_ID, TRIGGERING_TIME, VIEWED_COUNT
IN_APP_PAGER_COMPLETED: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE, APP_PACKAGE_NAME, APP_VERSION DEVICE_MODEL, DEVICE_OS, CARRIER, PUSH_OPT_IN, BACKGROUND_PUSH_ENABLED, LOCATION_ENABLED, LOCALE_LANGUAGE_CODE, LOCALE_COUNTRY_CODE, LOCATION_PERMISSION, IANA_TIMEZONE, APP_DEFINED_ID, COMPLETED, GROUP_ID, RENDERED_LOCALE, PAGE_COUNT, PAGE_IDENTIFIER, PAGE_INDEX, PAGER_IDENTIFIER, PUSH_ID, SESSION, TIME_SENT, TRIGGERING_PUSH_ID, TRIGGERING_GROUP_ID, TRIGGERING_VARIANT_ID, TRIGGERING_TIME
IN_APP_PAGER_SUMMARY: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE, APP_PACKAGE_NAME, APP_VERSION DEVICE_MODEL, DEVICE_OS, CARRIER, PUSH_OPT_IN, BACKGROUND_PUSH_ENABLED, LOCATION_ENABLED, LOCALE_LANGUAGE_CODE, LOCALE_COUNTRY_CODE, LOCATION_PERMISSION, IANA_TIMEZONE, APP_DEFINED_ID, COMPLETED, GROUP_ID, RENDERED_LOCALE, PAGE_COUNT, PAGE_IDENTIFIER, PAGER_IDENTIFIER, PUSH_ID, SESSION, TIME_SENT, TRIGGERING_PUSH_ID, TRIGGERING_GROUP_ID, TRIGGERING_VARIANT_ID, TRIGGERING_TIME
SEND_ABORTED: ID, TYPE, OCCURRED, PROCESSED, OFFSET, NAMED_USER_ID, ANDROID_CHANNEL, IOS_CHANNEL, AMAZON_CHANNEL, CHANNEL, DEVICE_TYPE, APP_PACKAGE_NAME, APP_VERSION DEVICE_MODEL, DEVICE_OS, CARRIER, PUSH_OPT_IN, BACKGROUND_PUSH_ENABLED, LOCATION_ENABLED, LOCALE_LANGUAGE_CODE, LOCALE_COUNTRY_CODE, LOCATION_PERMISSION, IANA_TIMEZONE, GROUP_ID, PUSH_ID, REASON

Feedback

Was this page helpful?