JSON Flex leverages the advantages of the Chaos Index® technology to uniquely represent the dimensionality of JSON by giving administrators the tools to selectively control how the JSON source files are indexed, creating the highly optimized Chaos Index data that drives query performance and information value for the end-user analysts.
ChaosSearch object grouping features give administrators more insight and control over how to select and flatten their JSON source files (pre-indexing), with options for specifying the fields to include, exclude, and the structure of the indexed data, either as sequences of horizontal columns or vertical rows, or fields of JSON strings with text-searchable content.
Chaos Refinery® features help to prepare user-ready views with the valuable columns for querying and analytics. Refinery views can also use several transformation options to extract information from the indexed data, such as virtually transforming horizontally flattened arrays into vertically extracted fields for filtering and aggregation, or materializing specific properties within JSON string fields, to create the columns that are valuable for the query and analytics needs of the users.
First, when you create an object group to focus on the JSON files that you want to index, you can specify whether to use horizontal or vertical expansion to balance data visibility with the indexed data schema size of rows and fields. See JSON Log Flattening Example for a general example of horizontal and vertical expansion effects. After you select an overall flattening policy for the object group files, you can then use object group schema transformations to tune the fields for the indexed data, as well as to apply more granular rules for storing nested fields or arrays as JSON strings to reduce the row and column widths and gain better query performance.
Second, when you create Refinery views for the object groups, you can take advantage of more JSON Flex features such as the JSON Array Transformation (to selectively expand horizontally flattened arrays into virtual, vertically expanded columns) to make those fields and their content available as filters and metrics for operational analysis. In addition, view transformation options can apply specific rules to indexed data fields to materialize columns for important data analysis and querying filters.
Another Look at JSON Flex
For a closer look at how a simple nested JSON file can quickly inflate with expansion operations, see the ChaosSearch blog: The New Best Way to Index and Query JSON Logs.
Updated 19 days ago