Singular to S3: Setup Guide

Follow the instructions below to let Singular to export data in a streamlined and automated manner to an Amazon S3 bucket.

Step 1 - Create an S3 Bucket

The first step is to create an Amazon S3 bucket that Singular will export data to. If you already have an S3 bucket set up, please skip to Step 2.

In order to create an Amazon S3 bucket, please follow these steps:

  1. Sign in to the AWS console.
  2. Go to Services -> S3.
  3. Click on "Create Bucket"
    mceclip1.png
  4. Choose a name and a region for your bucket (the region should typically be the same as the region in which you use other AWS services, such as EC2 or Redshift)
    mceclip3.png
  5. The rest of the settings can remain as the default settings (or changed according to your needs)
  6. Finally, click on "Create Bucket" in the last step.

Step 2 - Provide Singular with permissions to the bucket

There are two ways to set up access to your bucket:

Option 1 - Create a Bucket Policy (recommended)

In this option, you provide Singular's AWS account with direct access to your S3 bucket. You can accomplish that with the following steps:

  1. In the AWS console, go to Services -> S3.
  2. Click on the relevant S3 bucket.
  3. Click on the Permissions tab, and under it click on Bucket Policy.
    mceclip4.png
  4. Please add the following policy to your Bucket Policy (replace <YOUR_S3_BUCKET_NAME> with the real bucket name):
    {
      "Version": "2012-10-17",
      "Id": "",
      "Statement": [
        {
          "Sid": "SingularS3Access",
          "Effect": "Allow",
          "Principal": {
            "AWS": [
              "arn:aws:iam::623184213151:root"
            ]
          },
          "Action": [
            "s3:GetObject",
            "s3:PutObject",
            "s3:ListBucket"
          ],
          "Resource": [
            "arn:aws:s3:::<YOUR_S3_BUCKET_NAME>",
            "arn:aws:s3:::<YOUR_S3_BUCKET_NAME>/*"
          ]
        }
      ]
    }
  5. Finally, click Save to finish.

After completing the steps above, please provide Singular with the following information:

  • The S3 bucket name.

 

Option 2 - Provide Singular with an access key id / secret access key pair

Alternatively, you can manage permissions by creating a dedicated AWS user that has access (only) to the relevant S3 bucket, and providing Singular with an access key id / secret access key pair.

You can accomplish that with the following steps:

  1. In the AWS console, go to Services -> IAM.
  2. Go to the left menu, and click on "Policies".
  3. Click "Add Policy".
  4. Click on the "JSON" tab.
  5. Add the following policy (replace <YOUR_S3_BUCKET_NAME> with the real bucket name):
    {
      "Version": "2012-10-17",
      "Id": "",
      "Statement": [
        {
          "Sid": "SingularS3Access",
          "Effect": "Allow",
          "Action": [
            "s3:GetObject",
            "s3:PutObject",
            "s3:ListBucket"
          ],
          "Resource": [
            "arn:aws:s3:::<YOUR_S3_BUCKET_NAME>",
            "arn:aws:s3:::<YOUR_S3_BUCKET_NAME>/*"
          ]
        }
      ]
    }
  6. Click on Review Policy.
  7. Provide a name for the new policy (e.g. "singular-s3-exports") and click Create Policy.
  8. Go to the left menu, and click on Users.
  9. Click Add User.
  10. Choose a name for the user, and make sure to enable "Programmatic Access" (but do not enable console access):
    mceclip6.png
    And finally, click on Next: Permissions
  11. Under "Set Permissions", choose the option "Attach existing policies directly", and add the policy that was created in step (7):
    mceclip7.png
    And finally, click on Next: Tags.
  12. If needed, add relevant tags, otherwise click on Next: Review. If all looks good, click on Create User.
  13. Finally, save the newly created Access key ID and Secret access key:mceclip8.png
     

After completing the steps above, please provide Singular with the following information:

  • The S3 bucket name.
  • Access Key ID
  • Secret Access Key

 

Was this article helpful?
0 out of 0 found this helpful