Set up an S3 data destination to export data from Singular automatically to an Amazon S3 bucket.

Setup Instructions

1. Create an S3 Bucket

Create an Amazon S3 bucket to which Singular will export data.

1. In your AWS console, go to Services > S3 and click Create Bucket.

2. Enter a name for your bucket. The name must start with singular-s3-exports-, as Singular filters for buckets with this prefix. For example:

   singular-s3-exports-mycompanyname

3. Select an AWS region for your bucket. The region should typically be the same as the region in which you use other AWS services, such as EC2 or Redshift.
4. You can keep the default values for the rest of the settings.

2. Give Singular Permissions to the Bucket

There are two ways to set up access to your bucket:

Option A (Recommended): Create a Bucket Policy

You can provide Singular's AWS account with direct access to your S3 bucket as follows:

1. In the AWS console, go to Services > S3 and select the relevant S3 bucket.
2. Select the Permissions tab and select Bucket Policy.

3. Add the following to your Bucket Policy (replace <YOUR_S3_BUCKET_NAME> with the real bucket name):

   {
    "Version": "2012-10-17",
    "Id": "",
    "Statement": [
    {
    "Sid": "SingularS3Access",
    "Effect": "Allow",
    "Principal": {
    "AWS": [
    "arn:aws:iam::623184213151:root"
    ]
    },
    "Action": [
    "s3:GetObject",
    "s3:PutObject",
    "s3:ListBucket",
    "s3:GetBucketLocation",
    "s3:DeleteObject"
    ],
    "Resource": [
    "arn:aws:s3:::<YOUR_S3_BUCKET_NAME>",
    "arn:aws:s3:::<YOUR_S3_BUCKET_NAME>/*"
    ]
    }
    ]
   }

   Important: Singular sets ACL to bucket-owner-full-control to ensure the bucket owner has full access to uploaded files. Make sure your bucket policy does not deny PutObjectACL permissions to the Singular account!

Option B: Give Singular an Access Key ID + Secret Access Key

If you prefer, you can manage permissions by creating a dedicated AWS user that has access (only) to the relevant S3 bucket, and giving Singular an access key ID and secret access key.

1. In the AWS console, go to Services > IAM. In the menu on the left, click Policies.
2. Click Add Policy and click the JSON tab.
3. Add the following policy (replace <YOUR_S3_BUCKET_NAME> with the real bucket name):

   {
    "Version": "2012-10-17",
    "Id": "",
    "Statement": [
    {
    "Sid": "SingularS3Access",
    "Effect": "Allow",
    "Action": [
    "s3:GetObject",
    "s3:PutObject",
    "s3:ListBucket",
    "s3:GetBucketLocation",
    "s3:DeleteObject"
    ],
    "Resource": [
    "arn:aws:s3:::<YOUR_S3_BUCKET_NAME>",
    "arn:aws:s3:::<YOUR_S3_BUCKET_NAME>/*"
    ]
    }
    ]
   }

4. Click Review Policy and give the new policy a name (e.g. "singular-s3-exports").
5. Click Users > Add User.

6. Choose a name for the user and enable "Programmatic Access" (but do not enable console access):

   

7. Click Next: Permissions and under Set Permissions select Attach existing policies directly.

8. Add the policy you just created:

   

9. Finish creating the user and save the newly created access key ID and secret access key.

3. Add an S3 Data Destination

To add an S3 data destination in Singular:

1. In your Singular account, go to Settings > Data Destinations and click Add a new destination.
2. Type in either "S3 Destination" (to export aggregated marketing data) or "S3 User-Level Destination" (to export user-level data).

3. In the window that opens, fill in the bucket name you created:

   

4. If you created an access key ID and secret access key, select "Using AWS Access Key ID + AWS Secret Access Key" in the "Bucket Access Type" dropdown.

   If you gave Singular access using a bucket policy, you should leave "Using Bucket Policy".

   

5. Select an output format: "CSV" or "Parquet".

6. Choose an output path in your S3 Bucket, for example, "singular_marketing_data/{date}{extension}".

   Important: For the file partitions to be uploaded correctly, the final object in the output key pattern must be:

   • {date}{extension} for aggregate destinations
   • {timestamp}{extension} for user-level destinations
7. Choose the schema of the data loaded into the destination. For your schema options, see Data Destinations: Aggregated Marketing Data Schemas and Data Destinations: User-Level Data Schemas.

Placeholders

Singular supports the following placeholders (macros) that will get expanded automatically: