Chaos Data Advantages

ChaosSearch developed a patented data representation that automatically transforms raw data into a compact, lossless representation—providing 10-20x compression—compared to the raw data and other indexing solutions. The data is not just small, it is fast, and has intelligence, yielding much higher performance and capabilities for the analytics that use it.

Since indexed data is small, it can be written to and read from cloud storage with very low latency. It supports longer and even unlimited retention versus other solutions. Consequently, storage needs and costs for Chaos Index data is greatly reduced versus other solutions.

The small Chaos Index data also contributes to savings in compute and networking. Combined with the stateless/serverless ChaosSearch operations, ChaosSearch can achieve virtual memory speeds without the need for local caching in compute. With the ChaosSearch design, indexing and analysis work can be scheduled, run, and even terminate without affecting any other ongoing ingest or analysis tasks.

ChaosSearch takes full advantage of cloud compute and its inherent scalable, shareable, and manageable nature. Workloads for query and indexing can burst, lease, and steal worker resources based on policies and metrics. Expenses for compute and networking is greatly reduced versus other solutions with heavy memory footprints and longer-running tasks. And, more importantly, these operations offer faster turnaround to index the raw data, and faster query performance for the users.

The ChaosSearch data representation is not just small and fast, it is smart as well. Chaos Index data delivers schema-on-read flexibility with the power of schema-on-write performance. The data representation enables support for dynamic transformation (as in the "T" of extract-transform-load) by end users. Within the view definition, users can dynamically define a shape for the data they want to analyze, enabling transformations to meet specific needs or analysis types. ChaosSearch helps to reduce the time and expertise requirements for data engineers—and pipeline toil—to leverage the indexed data for specific analysis needs on the fly, and most importantly, without re-indexing.