Advanced Filtering

When you create an object group and specify the cloud storage files to index, you can use the optional Advanced Filtering button to use some stricter filtering rules for the object group. These filter features include options for including or skipping files based on modification date ranges, file size, or storage class.

This window also offers controls for specifying an isolation key to separate the indexed data based on keys derived from the rule. You can also apply object tags/metadata to the resulting indexed data rows.

The following sections provide more information on the Advanced Filtering options.

Cloud Storage Filtering Criteria

You can use the advanced filters to include or omit cloud storage files to index based on characteristics such as:

• Last Modified—Specify a date range to index only the cloud storage files that have a modification date inside the date range.
• Size (bytes)—Specify a file size range in bytes to index cloud storage files that have a size within the specified byte size range.
• Storage Class—Limit indexing to a specific cloud storage class/service like Standard, Glacier, or other to index only the files within that class.

Isolation Keys

You use the Isolate By field to define isolation key expressions (also called partition keys) that store the object group's indexed data into different segments to keep related data together and separate from segments of indexed data for other matching keys for the group. For the details on isolation keys, see Object Group Isolation Keys.

You can use the Isolate By (LastModified) field create to create segments of indexed data for object files that have the same modification date window. This feature improves scoping and query performance when the last modification time for a file closely matches the timestamps of the events within the file. If you isolate by a modification date window of 1 hour, for example, the indexed data for the files that have a modification date within the same hour will be stored together in the same segment. By storing the events for the same time frame in the same segment, time-based queries against that object group can run much faster.

The valid values for the Units when specifying Isolation By LastModified are:

• h hr hour
• m min minute

Remember that ChaosSearch creates daily index intervals by default, so a value greater than 24 hours will result in daily interval segments, which is the default. A value that is too small (say 1 minute) could create a very large number of segments, which would diminish the scoping and performance gain. It is recommended to work with Customer Success to plan the right window size for a good balance of segment counts and segment size based on the modification date patterns and the typical user query time ranges.

Object Tags/Metadata

You can also create object tags or metadata to apply to the indexed data. To create a tag or metadata value:

1. If needed, click Add Tag to create an editable row for a new tag or metadata value.
2. In the Type field, select Tag (the default) or Metadata. All rows that you add will be of the selected type.
3. In the Key field, type a name for the tag or metadata.
4. In the Value field, type a value for the tag or metadata.

📘

Remember to click Save.

After you specify one or more of the advanced filtering options, make sure to click Save to save the settings and close the window.

Click Reset Filters to clear the window and any values that have been specified in any fields.