Advanced Filtering

When you create an object group and specify the cloud storage files to index, you can use the optional Advanced Filtering button to define some stricter filtering rules for the object group. These filter features include options for including or skipping files based on modification date range, file size, or storage class.

Within this window, you can specify an isolation key rule to separate the indexed data based on keys derived from the rule. You can also apply object tags/metadata to the resulting indexed data rows.

The following sections provide more information on the Advanced Filtering options.

Cloud Storage Filtering Criteria

You can use the advanced filters to include or omit cloud storage files based on characteristics such as:

• Last Modified—Specify a date range to index only the cloud storage files that have a modification date inside the range.
• Size (bytes)—Specify a file size range in bytes to omit cloud storage files that are outside the specified size range.
• Storage Class—Limit indexing to a specific cloud storage class/service like Standard, Glacier, or other to index only the files in that class.

Isolation Keys

You use the Isolate By field to define isolation keys that can create separate collections of indexed data for the object group's indexed data. For the details on isolation keys, see Object Group Isolation Keys.

You can use the Isolate By (LastModified) field create to create segments of indexed data for object files that have the same modification date window. This feature improves scoping and query performance when the last modification time for a file closely matches the timestamps of the events within the file. If you isolate by a modification date window of 1 hour, for example, the indexed data for the files that have a modification date within the same hour will be stored together in the same segment. By storing the events for the same time frame in the same segment, time-based queries against that object group can run much faster.

The valid values for the Units when specifying Isolation By LastModified are:

• h hr hour
• m min minute

Remember that ChaosSearch creates daily index intervals by default, so a value greater than 24 hours will result in daily interval segments. A value that is too small (say 1 minute) could create a very large number of segments, which would diminish the scoping and performance gain. It is recommended to work with Customer Success to plan the right window size for a good balance of segment counts and segment size based on the modification date patterns and the typical user query time ranges.

Object Tags/Metadata

You can also create object tags or metadata to apply to the indexed data. To create a tag or metadata value:

1. If needed, click Add Tag to create an editable row for a new tag or metadata value.
2. In the Type field, select Tag (the default) or Metadata. All rows that you add will be of the selected type.
3. In the Key field, type a name for the tag or metadata.
4. In the Value field, type a value for the tag or metadata.

📘

Remember to click Save.

After you specify one or more of the advanced filtering options, make sure to click Save to save the settings and close the window.

Click Reset Filters to clear the window and any values that have been specified in any fields.