Data Export Guide

Updates to Data Export file retention policy Starting 25 May 2018, to comply with the upcoming GDPR privacy requirements and to enhance your control over your data, Optimizely will retain the files in your Data Export bucket for 30 days. To keep your file history over 30 days, update your import process to archive your files at least once every 30 days.


Data Export allows developers to access all of their Optimizely event data. This data is computed daily and contain the last 24 hours of events for all A/B testing experiments in an account. The data will be written securely to an S3 bucket, which you can then programmatically access via Amazon’s APIs and a set of secure credentials provided by Optimizely.

You can use Data Export to access Optimizely experiment data with your own data warehouse.

If you have any questions, you can ask the developer community, or you can submit a ticket to the developer support team. We'll be happy to assist you.

Availability

Currently, this feature is available to Enterprise customers; please reach out to your Customer Success Manager if you wish to utilize this feature. If you do not have a CSM, submit a ticket to the developer support team to verify your plan and eligibility.

Technical Details

Data is written out for all experiments in all projects running under an Optimizely account, and one file will be written per experiment per day. Each file will contain 24 hours worth of data, thru midnight UTC of the previous night. These files are gzipped tab-delimited files with the format described below. Also, if you currently receive a data export via the Technical Support team, the data will be in the same format as the current exports, with the addition of one column indicating user_agent for web experiment data. There is also a standard header row.

The S3 bucket location will follow the format: /optimizely-export/{account_id}/{project_id}/yyyy/mm/dd/{file_name}

The file name follows the format: experiment_id-yyyy-mm-dd.tsv.gz . Example: 123-2016-03-20.tsv.gz

Data Export Chart

Note: Partitioning of results will occur for very large experiments. 40 different files will be produced for one experiment. E.g. instead of 7473240577-2016-10-15.tsv.gz the file names will be 7473240557-<index>-2016-10-15.tsv.gz where <index> ranges from 0-39.

Web Dictionary

Sample File

Definitions

timestamp

The timestamp of when the event was received, not necessarily when it occurred in the browser. The format is YYYY-MM-DDTHH24:MI:SS.sssZ (ISO format), and the timezone is UTC.

project_id

Your Optimizely project ID on which the experiment lives.

experiment_id

The experiment ID.

variation_id

The id we use to identify the variation the user saw. This should correspond to the variation id in the Diagnostic Report in the Options menu of the Visual Editor.

end_user_id

This is the anonymous optimizelyEndUserId cookie value. It represents a unique visitor.

uuid

Similar in scope to end_user_id, but this is the customer's API-provided unique ID they want to track in lieu of the optimizelyEndUserId cookie value. This was renamed from the legacy ppid on Thursday, 2016-05-19.

user_ip

IP of the user associated with this tracking call.

user_agent

User-Agent header passed from the browser.

revenue

If applicable, the amount of the transaction in cents (399 corresponds to $3.99). This will only be populated with a non-zero value for revenue goals.

event_name

The event name of the tracking call. For pageviews this is the URL of the page; for the default Engagement goal this is 'engagement'; and for everything else this is the custom event name. When an experiment activates, if there are matching pageview goals (project-wide), there will be one row with a value of 'optly_activate' and one row with the URL per pageview goal. If there are no matching pageview goals, there will be one row with the URL of the page where the experiment was activated.

mobile visitors

For accounts with segmentation a true or false value of whether or not the user was using a mobile device (this is an Optimizely default segment, so tablets are considered mobile here).

browser

For accounts with segmentation this is the browser that was being used by the user when the tracking call was made. gc is Google Chrome, ff is Firefox, and ie is Internet Explorer. safari, opera, and ucbrowser are listed as-is.

source type

For accounts with segmentation this is the value of the traffic source that the user falls into (campaign, direct, referral, search).

campaign

The value of the campaign segment (i.e. AdWords utm_campaign parameter value) tied to this user. Default value is "none." Any fields after this are your segmented audiences or dimension names.

Please Note:

Android Dictionary

Sample File

Definitions

timestamp

The timestamp of when the event was received, not necessarily when it occurred in the browser. The format is YYYY-MM-DDTHH24:MI:SS.sssZ (ISO format), and the timezone is UTC.

project_id

Your Optimizely project ID on which the experiment lives.

experiment_id

The experiment ID.

variation_id

The id we use to identify the variation the user saw. This should correspond to the variation id in the Diagnostic Report in the Options menu of the Visual Editor.

end_user_id

This is the anonymous optimizelyEndUserId value. It represents a unique visitor.

uuid

Similar in scope to end_user_id, but this is the customer's API-provided unique ID they want to track in lieu of the anonymous value. This was renamed from the legacy ppid on Thursday, 2016-05-19.

user_ip

IP of the user associated with this tracking call.

revenue

If applicable, the amount of the transaction in cents (399 corresponds to $3.99). This will only be populated with a non-zero value for revenue goals. For event_name values of "mobile_session," this will also show the length of the session in seconds..

event_name

The event name of the tracking call. A value of "mobile_session" means a new session was recorded (a session is a period of activity during which the app is foregrounded, without a break longer than 30 seconds). A value of "visitor-event" shows the first time a visitor sees an experiment. A tap or view goal will show the view name with #tap or #view appended. For everything else it is the custom event name.

Android App Version

The numeric version of your Android app running on the user’s device.

Android SDK Version

The numeric version of Optimizely’s Android SDK running in your app on the user’s device.

Android Device Model

The device’s name as returned to Optimizely’s SDK, e.g. Google Galaxy Nexus - 4.2.2 - API 17 - 720x1280.

iOS Dictionary

Sample File

Definitions

timestamp

The timestamp of when the event was received, not necessarily when it occurred in the browser. The format is YYYY-MM-DDTHH24:MI:SS.sssZ (ISO format), and the timezone is UTC.

project_id

Your Optimizely project ID on which the experiment lives.

experiment_id

The experiment ID.

variation_id

The id we use to identify the variation the user saw. This should correspond to the variation id in the Diagnostic Report in the Options menu of the Editor.

end_user_id

This is the anonymous optimizelyEndUserId value set in NSUserDefaults. It represents a unique visitor.

uuid

Similar in scope to end_user_id, but this is the customer's API-provided unique ID they want to track in lieu of the anonymous value. This was renamed from the legacy ppid on Thursday, 2016-05-19.

user_ip

IP of the user associated with this tracking call.

revenue

If applicable, the amount of the transaction in cents (399 corresponds to $3.99). This will only be populated with a non-zero value for revenue goals. For event_name values of "mobile_session," this will also show the length of the session in seconds.

event_name

The event name of the tracking call. A value of "mobile_session" means a new session was recorded (a session is a period of activity during which the app is foregrounded, without a break longer than 30 seconds). A value of "visitor-event" shows the first time a visitor sees an experiment. A tap or view goal will show the view name with #tap or #view appended. For everything else it is the custom event name.

iOS App Version

The numeric version of your iOS app running on the user’s device.

iOS Device Model

The device’s name as returned to Optimizely’s SDK, e.g. iPhone.

iOS SDK Version

The numeric version of Optimizely’s iOS SDK running in your app on the user’s device.

Please Note: (web, iOS & Android)

Data formats

As these files are TSVs, nulls will be empty tabs.

Primary Keys

Status File

A status file will be provided to track the success or failure of that day's experiment event files. This file is named status.yaml and is included in the daily folder per project. The format contains: failed_exports, successful_exports, and a timestamp in UTC seconds since epoch. View a sample YAML file with and without failed export files.

Additional Notes:

Miscellaneous