If you currently use, or plan to use, AWS S3 buckets to store your data files for querying, follow these steps to provide ChaosSearch indexing services with access to your S3 buckets.

You can perform these connection and access steps in the following ways:

• Manually from the AWS Identity and Access Management (IAM) interface (described in this topic)
• Using a CloudFormation template
• Using an orchestration tool such as Terraform

IAM Access Instructions

Use this method if you are very familiar with the AWS IAM interfaces and screens. If you are not sure about using IAM, use the CloudFormation method instead.

🚧

NOTE

There are multiple versions of the AWS console and the order/terminology of following steps could be different for your version. These steps are intended as an overview of how to configure access roles and permissions.

Create a New Role for ChaosSearch in AWS

📘

AWS IAM Role Requirements

ChaosSearch uses Amazon AWS IAM roles to allow you to grant access to your S3 buckets to the ChaosSearch service. Follow these steps to configure an AWS account.

1. Log in to your AWS console. If you do not already have an AWS account, see create an AWS account.
2. Navigate to the Roles page of the IAM service (Services > Security, Identity & Compliance > IAM > Roles).

3. Click Create role.
4. In the Create role window, click Another AWS account.

5. Type the ChaosSearch AWS account ID (515570774723) in the Account ID field
6. Check/select the Require external ID option.
7. In the External ID field, paste or type the external ID for your ChaosSearch account. You can obtain the external ID from your ChaosSearch portal Settings/Help page for the AWS Credentials.
8. Click Next: Permissions. The Create policy page appears.

9. Proceed to the next section to define policy permissions for the role.

IAM Role Policy Permissions

Follow these steps to grant ChaosSearch read access to the AWS storage buckets. You can configure different access policies, such as:

• Access to all buckets in storage
• Access to a specific bucket (recommended)
• Access to buckets that share a common prefix name value

👍

NOTE:

If you want to grant full access to ChaosSearch, you can assign one of the AWS predefined policies instead of creating a custom policy for more specific bucket access.

For example, you could attach the “AmazonS3FullAccess” policy to your custom ChaosSearch role, skip the section for creating a custom policy, and proceed to the following Attach the Policy section.

🚧

CHAOSSEARCH S3 Bucket Access

For ChaosSearch to read the bucket(s) that you want to start indexing, ChaosSearch must be able to discover and list all of the S3 buckets. If you have any questions, contact the ChaosSearch Team.

Read-only Permissions – All Buckets

1. In the Create policy page, click the JSON tab.

2. In the JSON code area, replace or append the starter code with the policy that you want to use for your AWS buckets. For example, a read-only policy for all buckets follows. (Two other policy examples are described in the following sections.)

{
    "Version": "2012-10-17",
    "Statement": [
            {
            "Effect": "Allow",
            "Action": [
                "s3:Get*",
                "s3:List*"
            ],
            "Resource": "*"
            },
            {
            "Effect": "Allow",
            "Action": "*",
            "Resource": [
                "arn:aws:s3:::cs-EXTERNAL-ID",
                "arn:aws:s3:::cs-EXTERNAL-ID/*"
            ]
            }
    ]
}

The following window shows the policy window for read-only permission on all buckets. If you use this sample code, be sure to replace EXTERNAL-ID with the external ID for your AWS account.

3. Click Next: Tags. The Add Tags (optional) page appears.
4. Click Next: Review. The Review policy page appears.

5. In the Name field, type a name for the policy. A common sample name is 'ChaosSearchReadOnly'.
6. Type a brief description for the policy (optional).
7. Click Create policy. The system creates the new policy and displays the Policies page.

The next step is to attach the policy to the ChaosSearch role that you created previously.

Read-only Permissions – Specified Bucket (Recommended)

As an alternative policy, the following example is a JSON definition for a policy that applies read-only permissions to a specific bucket. (Be sure to replace EXTERNAL-ID with the external ID for your account.)

{
   "Version": "2012-10-17",
   "Statement": [
       {
           "Effect": "Allow",
           "Action": [
               "s3:ListAllMyBuckets",
               "s3:GetBucketLocation"
           ],
           "Resource": "*"
       },
       {
           "Effect": "Allow",
           "Action": [
               "s3:List*"
           ],
           "Resource": "arn:aws:s3:::airtest-testdata"
       },
       {
           "Effect": "Allow",
           "Action": [
               "s3:Get*",
               "s3:PutObjectTagging"
           ],
           "Resource": "arn:aws:s3:::airtest-testdata/*"
       },
       {
           "Effect": "Allow",
           "Action": "*",
           "Resource": [
               "arn:aws:s3:::cs-EXTERNAL-ID",
               "arn:aws:s3:::cs-EXTERNAL-ID/*"
           ]
       }
   ]
}

Read-only Permissions – Prefix-Specific Buckets

As an alternative policy, the following example is a JSON definition for a policy that applies read-only permissions to bucket names that begin with the same prefix. (Be sure to replace EXTERNAL-ID with the external ID for your account.)

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets",
                "s3:GetBucketLocation"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:List*"
            ],
            "Resource": "arn:aws:s3:::SOURCE_BUCKET"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:Get*",
                "s3:PutObjectTagging"
            ],
            "Resource": "arn:aws:s3:::SOURCE_BUCKET/some/path/prefix/*"
        },
        {
            "Effect": "Allow",
            "Action": "*",
            "Resource": [
                "arn:aws:s3:::cs-EXTERNAL-ID",
                "arn:aws:s3:::cs-EXTERNAL-ID/*"
            ]
        }
    ]
}

KMS Encryption

📘

KMS Encryption

If the Objects stored in S3 are encrypted, there is no need to decrypt these objects. ChaosSearch can naturally work with these objects when the appropriate resources are defined in the IAM Policy. An example of a policy for encrypted buckets follows.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "kms:GenerateDataKey",
                "kms:Decrypt"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:kms:us-east-1:123456789012:key/enter-key-here",
                "arn:aws:kms:us-east-1:123456789012:key/enter-key-here",
                "arn:aws:kms:us-east-1:123456789012:key/enter-key-here",
                "arn:aws:kms:us-east-1:123456789012:key/enter-key-here"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:Get*",
                "s3:List*",
                "s3:PutObjectTagging"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "*",
            "Resource": [
                "arn:aws:s3:::cs-s3bucketname",
                "arn:aws:s3:::cs-s3bucketname"
            ]
        },
        {
            "Effect": "Allow",
            "Action": "*",
            "Resource": [
                "arn:aws:s3:::cs-s3bucketname/*",
                "arn:aws:s3:::cs-s3bucketname/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": "*",
            "Resource": [
                "arn:aws:s3:::cs-*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": "*",
            "Resource": [
                "arn:aws:s3:::cs-*/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": "sqs:*",
            "Resource": [
                "arn:aws:sqs:*:123456789012:s3-sqs"
            ]
        }
    ]
}

In the sample policy, note the following code constructions:

• The first statement provides ChaosSearch with access to the Get, List, and PutObjectTagging operations (read-only access).

Note: "s3:PutObjectTagging" is optional. It is required only to enable object tagging in the ChaosSearch UI.

• The second statement provides a bucket for ChaosSearch to write its metadata and statistics. The name of the bucket must be named "cs-".

Attach the Policy to the ChaosSearch IAM Role

1. On the Roles page, select your newly created role to display it. The role Summary page appears.

2. Click Attach policies. The Add permissions to role page appears.
3. Type your new policy name into the Policies type filter. For example, "ChaosSearchReadOnly".
4. Select the custom policy (or an AWS policy if you prefer) and click Attach policy.

The role Summary page appears and your new policy is attached to the role.

Add Role ARN to ChaosSearch AWS Credentials

After you create a role for ChaosSearch and add the required policies for the role to access your S3 buckets, follow these steps to update the AWS Credentials page with the role ARN.

1. In the AWS IAM console Roles page, click the newly created role to display its information.
2. Copy the value in the Role ARN field.

3. Navigate to the ChaosSearch console and open the Settings/Help page.
4. Paste the role ARN into the Role ARN field.
5. Click Update.
6. Make sure that the check mark icon appears to confirm that the role ARN is valid.

🚧

Troubleshooting S3 Access

If the ChaosSearch console displays 'Request Failed' messages when you select a bucket or object group in the Storage window, review the following documentation for possible troubleshooting: AWS doc.