Configuring Object Group Isolation Keys

Review the steps in this topic to create an object group that can use isolation keys to separate data into different slices. Make sure that you have a pathname/regular expression to identify the target cloud storage files for the object group, and make sure that you have the regular expression that will define the keys used to separate the index data into the different isolation slices for querying.

Create an Object Group That Uses Isolation

To create an object group that uses isolation keys:

1. Follow the standard instructions to create an object group. Go to Storage and then click Create Object Group.
2. In the Prefix and/or RegEx Filter field, type the values to filter the cloud object files that you want this object group to index.

In this example, the cloud object storage contains application log files that are generated for different regions in the cloud service. The files have pathnames in the format cloudtrail/AWSLogs/id/<region>/<date>/<filename>.json.gz where <region> is the cloud service region, such as ap-northeast-1, -2, and -3. (This example will create isolation keys based on the platform region folder names.)

3. Click Advanced Filtering to display the filter options.

4. In the Isolate By field, type the regular expression that you want to use for the isolation key value, or click the pencil icon to display a regular expression editor window in which you can construct and test a regular expression. A sample expression editor window follows. The Content Preview list shows up to 50 of the cloud-storage files that match the object group prefix and filter values.

5. In the Regex field, review or type the regular expression to use for isolation. This example uses the expression cloudtrail/AWSLogs/250/CloudTrail/(\S+?)/20.* to isolate log files by the region value.

In the Capture Groups area below the preview, look for a Match statement with one of the values (usually the first found value) derived from the regular expression. The filter column will be cs_partition_0 as denoted by the 0 before the ap-northeast-1 in the Capture Groups area. If the Capture Groups area is empty, update the isolation regex to make sure that it is correct.

6. Complete the usual steps to create an object group by specifying its name, live indexing information, and options.

As shown in the sample window, the object group file filter and the isolation key results in three keys called ap-northeast-1, ap-northeast-2, and ap-northeast-3 based on the path names found in the cloud object storage. A sample object group Isolation tab follows:

The live object group watches for any new ap-northeast-* files added to cloud storage and indexes those files as they are made available. If a new region like ap-northeast-4 is added over time, a new key is created for that region. In the object group fields, ChaosSearch creates a cs_partition_key_0 field that can be used to reference the isolation keys for view filtering.

Use a Named Capture Group for the Isolation Key Column

If you do not want the default cs_partition_key_# column name as the isolation key, you can use a named capture group regex to specify an isolation key column name using the syntax ?<namestring> in this style:

cloudtrail/AWSLogs/250/CloudTrail/(?<RegionFilter>\S+?)/20.*

The isolation key column will be RegionFilter as denoted by the string before the ap-northeast-1 in the Capture Groups area above. Make sure that a capture group name is unique within the object group schema; for example, the string RegionFilter cannot match another column name in the same object group schema.

📘

Watch out for regex strings that could result in both named and cs_partition_# capture group column names.

Make sure that an object group with isolation has only named capture group isolation columns, or only cs_partition_# isolation columns, but not a mix of both types.