Index Lifecycle Management

ChaosSearch Index Lifecycle

The Chaos Index process creates daily intervals when it indexed the cloud storage objects for an object group, and it saves the daily intervals in your designated storage bucket. These daily intervals manage the indexed data that is used by the ChaosSearch features for queries, visualizations, and analytics.

Lifecycle management allows administrators to define how long to keep the daily intervals (and related indexed data) for each object group to automate the retention and cleanup of old daily intervals. You can define the retention period to set an age-out time that controls when the daily intervals are removed.

🚧

NOTE:

The aging process uses the date in the name of the daily interval, which is the modification date for the cloud storage files that are indexed in that interval for an object group.

Setting a Retention Policy

Lifecycle management is defined at the end of object group creation, as described in Creating Object Groups. Administrators can select the Retention Policy and define when the daily intervals should be deleted. Time frames can be specified as a number of days or months.

Administrators have the option to retain an object group's daily intervals "forever" without an automated cleanup. Deselect the Retention Policy field to specify an Unlimited retention of daily intervals.

Modifying the Retention Policy

After an object group is created, administrators can modify the retention policy settings to adjust the lifecycle for the daily intervals. Changes to the policy can either increase or decrease the aging out time range of the indexes associated with the object group.

❗️

CAUTION:

If you modify a Retention Policy to decrease/shorten the lifecycle period below its current value, the system immediately deletes any daily intervals for the object group if the interval name has a date that is older than the lifecycle range that you specified.

To modify the interval lifecycle setting:

1. In the Storage area, select the object group for which you want to change the interval lifecycle.
2. Click the Properties tab.
3. Click the current value for the Retention Policy. The Retention Policy window opens.

4. Specify the new desired time period for the policy. You can select a new number of days/months as applicable. You can also change to/from Unlimited retention as needed. When Unlimited is selected, you cannot edit the Overall number and Time Unit fields.
5. Click View Full Retention Policy to display the retention settings as a JSON file. For example:

6. Click Save to place the new retention period into effect.

When the Retention Policy range is updated, the system displays a confirmation window when the changes could result in the deletion of daily intervals that have dates before the new retention period value. Click OK to confirm that the new retention period is correct and that any daily intervals for prior days should be removed.

The interval cleanup is a process that runs on a periodic basis to remove intervals that have aged beyond the group's retention policy time. It can take several minutes before old intervals are removed. After intervals are cleaned up and removed, the Events tab for the object group displays a message that the _interval_file_ exceeded the interval retention to highlight when an interval has been removed because it exceeded a retention policy.

Checking Daily Interval Cleanup Events

When ChaosSearch runs a cleanup to remove old daily intervals, a message is added to the Events tab of the object group. If you have a scheduled retention policy, you can check the Events tab to confirm that the system is removing old daily intervals that are earlier than the retention period.