ChaosSearch Scroll API - Python

This section shows an example scroll API request

Scroll API Request

A search request through the ChaosSearch API returns a single page of results. The scroll API can be used to retrieve large numbers of results (or even all results) from a single search request.

📘

Scrolling is not intended for real-time user requests, but rather for processing large amounts of data.

Example Request

Fields to update:

  • Input your Access-Key-ID and Secret-Access-Key from Settings. (line 5)
  • Update the Index-View-Name with the one you created in the Refinery. (line 32)
  • Use the scroll field to specify how often you want the scroll API to move through results (for example, 1 minute = 1m). (line 34)
  • Use the size field to specify the number of results to return each time the scroll API makes a request. (line 35)
from opensearchpy import OpenSearch, helpers, exceptions, RequestsHttpConnection
import json
from requests_aws4auth import AWS4Auth

awsauth = AWS4Auth("Access-Key-ID", "Secret-Access-Key", "us-east-1", 's3')
os = OpenSearch(
  hosts = [{'host': 'lab.chaossearch.io', 'port': 443, 'url_prefix': '/elastic', 'use_ssl': True}],
  http_auth=awsauth,
  connection_class=RequestsHttpConnection,
  verify_certs=True
)

client = os

try:
    client_info = OpenSearch.info(client)
    print ('OpenSearch client info:', json.dumps(client_info, indent=4))
except exceptions.ConnectionError as err:
    print ('Opensearch client error:', err)
    client = None

if client != None:
    search_body = {
        "size": 500,
        "query": {
            "match_all": {}
        }
    }

resp = helpers.scan(
        client,
       index = 'Index-View-Name', # <---- SPECIFY ChaosSearch Refinery View HERE
        query = search_body,
        scroll = '1m',  
        size = 10,
    )

for num, doc in enumerate(resp):
        print ('\n', num, '', doc)

Did this page help you?