AWS S3 API

Examples from the AWS S3 API Reference Guide. Each section has the corresponding link for you to review further. For any questions, please contact CHAOSSEARCH Support.

CHAOSSEARCH introduces several new concepts for refining and querying data from your S3 buckets. Think of each of these concepts as phases in the data analysis flow, to ultimately gain insights and value.

CHAOSSEARCH currently supports Amazon S3 REST API calls for:

  • GET Service (ListAllMyBuckets)
  • GET Bucket (List Objects) Version 2
  • GET Object

GET Service (Specific Bucket)

https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGET.html

Returns some or all (up to 1,000) of the objects in a bucket

GET /V1/chaos-example-docs-bucket?list-type=2&delimiter=%2F&max-keys=100 HTTP/1.1
Host: subaccount.chaossearch.io
x-amz-date: Fri, 28 Feb 2019 21:32:02 GMT
Authorization: AWS ITKEVINDAVISEXAMPLEWKF:woKfdm36sGS+r3gF8KEM7IJ70=
x-amz-chaossumo-route-token: 5a31958b-fe3d-3m30-f00d-1da790bceee6
BucketName: "chaos-example-docs-bucket"
DiscoverData: {Complete: true, TrendingPrefixes: ["ip-127-0-0-1/"], Throttled: false,}
Complete: true
DuplicateStats: []
FileTypes: [{Type: "application/appd-stream", TotalFiles: 4, TotalSize: 327916, TotalTime: 6204714853000}]
0: {Type: "application/appd-stream", TotalFiles: 4, TotalSize: 327916, TotalTime: 6204714853000}
SecurityStats: [{Type: "Public Objects", Stats: 0,}, {Type: "Policies Allowed", Stats: 0,},]
0: {Type: "Public Objects", Stats: 0,}
DetailsURL: "s3://cs-5a31958b-fe3d-3m30-f00d-1da790bceee6/cs-data/stats/2019-03-12T14-26-29.114Z.chaos-example-docs-bucket.policy.objects.txt:52"
Stats: 0
Type: "Public Objects"
1: {Type: "Policies Allowed", Stats: 0,}
DetailsURL: "s3://cs-5a31958b-fe3d-3m30-f00d-1da790bceee6/cs-data/stats/2019-03-12T14-26-29.114Z.chaos-example-docs-bucket.policies.allowed.txt:55"
Stats: 0
Type: "Policies Allowed"
2: {Type: "Bucket Policies", Stats: 0,}
DetailsURL: "s3://cs-5a31958b-fe3d-3m30-f00d-1da790bceee6/cs-data/stats/2019-03-12T14-26-29.114Z.chaos-example-docs-bucket.bucket.policies.txt:54"
Stats: 0
Type: "Bucket Policies"
StartTime: 1552400789114
Throttled: false
TotalFiles: 4
TotalSize: 327916
TrendingPrefixes: ["ip-127-0-0-1/"]
0: "ip-127-0-0-1/"
id: "4478e5c9-hi42-ru81-or73-882f3c81f00d"

GET Service (List Buckets)

GET / HTTP/1.1
Host: subaccount.chaossearch.io
x-amz-date: Fri, 28 Feb 2019 21:32:02 GMT
Authorization: AWS ITKEVINDAVISEXAMPLEWKF:woKfdm36sGS+r3gF8KEM7IJ70=
x-amz-chaossumo-route-token: 5a31958b-fe3d-3m30-f00d-1da790bceee6

GET Bucket (List Objects) Version 2

https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGET.html

Operation returns some or all (up to 1,000) of the objects in a bucket

GET / HTTP/1.1
Host: subaccount.chaossearch.io
x-amz-date: Fri, 28 Feb 2019 21:32:02 GMT
Authorization: AWS ITKEVINDAVISEXAMPLEWKF:woKfdm36sGS+r3gF8KEM7IJ70=
x-amz-chaossumo-route-token: 5a31958b-fe3d-3m30-f00d-1da790bceee6

GET Object

http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectGET.html

To list metadata about all of the versions of objects in a bucket

GET /my-file.gz HTTP/1.1
Host: subaccount.chaossearch.io
x-amz-date: Fri, 28 Feb 2019 21:32:02 GMT
Authorization: AWS ITKEVINDAVISEXAMPLEWKF:woKfdm36sGS+r3gF8KEM7IJ70=
{
  "Bucket": "Cloudwatch Logs",
  "ModelStatus": {
    "parsed": {
      "rowCount": 60438,
      "analyzedSizeInBytes": 8990412
    },
    "objects": {
      "total": {
        "count": 316,
        "sizeInBytes": 911886,
        "first": {
          "etag": "\"362efbc187223423lkjslgkja34523\"",
          "size": 4220,
          "lastModified": "2019-02-28T22:20:19.000Z",
          "storageClass": "STANDARD",
          "key": "cs_query_alerting/2019022517201551392417/24234284-d09a-44d2-b0aa-1739f023erf3/2019-02-28-[$LATEST]f02986b6bfb34298b6aa89ad2dcdf8ee/000000.gz"
        },
    "pending": {
      "tasks": 0,
      "register": 0
    }
  },
  "Schema": {
    "fields": [
      {
        "id": "23342kl23-bacd-45a4-8b3f-7396c93651b1",
        "dataset": "79b86f69-10ff-4b13-85e4-658864891682",
        "pos": 0,
        "_type": "hidden"
      },
      {
        "_type": "direct",
        "pos": 1,
        "label": [
          "timestamp"
        ],
        "ref": 0,
        "id": "23342kl23-bacd-45a4-8b3f-7396c93651b1",
        "dataset": "79b86f69-10ff-4b13-85e4-658864891682",
        "loc": 1,
        "type": "TIMEVAL"
      },
      {
        "_type": "direct",
        "pos": 2,
        "label": [
          "message"
        ],
        "ref": 1,
        "id": "23342kl23-bacd-45a4-8b3f-7396c93651b1",
        "dataset": "79b86f69-10ff-4b13-85e4-658864891682",
        "loc": 2,
        "type": "STRING"
      }
    ],
    "orderings": [
      {
        "dID": "03651870233049146189",
        "first": {
          "field": {
            "id": "023342kl23-bacd-45a4-8b3f-7396c93651b1",
            "dataset": "23342kl23-bacd-45a4-8b3f-7396c93651b1",
            "pos": 0,
            "_type": "hidden"
          },
          "order": "asc"
        },
        "rest": [],
        "_type": "spec"
      }
    ]
  }
}

Common Request Headers

AWS S3 API


Examples from the AWS S3 API Reference Guide. Each section has the corresponding link for you to review further. For any questions, please contact CHAOSSEARCH Support.

Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.