Configuring Object Group Isolation Keys

Review the steps in this topic to create an object group that can use isolation keys to separate data into different slices. Make sure that you have a pathname/regular expression to identify the target cloud storage files for the object group, and make sure that you have the regular expression that will define the keys used to separate the index data into the different isolation slices for querying.

Create an Object Group That Uses Isolation

To create an object group that uses isolation keys:

1. Follow the standard instructions to create an object group. Go to Storage and then click Create Object Group.
2. In the Prefix and/or RegEx Filter field, type the values to filter the cloud object files that you want this object group to index.

In this example, the cloud object storage contains application log files that are generated for different regions in the cloud service. The files have pathnames in the format cloudtrail/AWSLogs/id/<region>/<date>/<filename>.json.gz where <region> is the cloud service region, such as ap-northeast-1, -2, and -3. (This example will create an isolation key based on the platform region folder names.)

3. Click Advanced Filtering to display the filter options.

4. In the Isolate By field, type the regular expression that you want to use for the isolation key value, or click the pencil icon to display a regular expression editor window in which you can construct and test a regular expression. A sample expression editor window follows.

5. In the Regex field, review or type the regular expression to use for isolation. This example uses the expression cloudtrail/AWSLogs/250/CloudTrail/(\S+?)/20.* to isolate log files by the region field.

Based on the object group filter, which included only the files that matched a pathname to files named ap-northeast*, the isolation keys will be ap-northeast-1, -2, and -3 for this example).

In the Capture Groups area below the preview, look for a Match statement with one of the isolation key values (usually the first found value) derived from the regular expression. If the Capture Groups area is empty, update the isolation regex to make sure that it is correct.

6. Complete the usual steps to create an object group by specifying its name, live indexing information, and options.

As shown in the sample window, the object group file filter and the isolation key results in three keys called ap-northeast-1, ap-northeast-2, and ap-northeast-3 based on the path names found in the cloud object storage. A sample object group Isolation tab follows:

The live object group watches for any new ap-northeast-* files added to cloud storage and indexes those files as they are made available. If a new region like ap-northeast-4 is added over time, a new key is created for that region.