JSON Flex Behavior Summary

JSON Flex offers a combination of pre- and post-indexing settings and features that can help to efficiently index even complex JSON log files—all without changing the raw JSON files. The options offer a combination of advantages and impacts.

As a general best practice, it can be very helpful to test some sample JSON files and try out the JSON Flex options to see which expansion and flex options might yield the best results for indexing and your analytics users.

Start with some representative JSON data (perhaps a few hours of JSON data) and use the ChaosSearch indexing features to create object groups with different options so that you can assess storage choices and differences. Similarly, you can create some test views for your sample object groups to review the indexed data results, to see the types of filters and columns that users will be able to access and use, and to try out the various JSON transformations that might create just the right level of filtering and visualization in the analytics phase.

Object Group Setting	Refinery View Setting	Pro's	Con's
Vertical Expansion		All JSON properties can be used as filters for analysis and reporting.	Usually the largest storage requirements for the indexed data for complex nested arrays. Some JSON fields might not be valuable for analysis filtering and could result in a large list to scan.
Horizontal Expansion	In the view, JSON Array Transformation can be used to select horizontally expanded arrays and virtually, vertically expand them to make their properties available as search filters.	Horizontal expansion is usually the best option for the smallest indexed data storage footprint for complex nested arrays.	Horizontally expanded nested attributes are usually not well suited for filtering or aggregate analysis.
Horizontal with vertical expansion whitelist. Hybrid of horizontal expansion; by default all arrays are horizontally expanded during indexing except the arrays referenced in the `vertical_selection_policy`.	Can use JSON Array Transformation for the horizontal arrays found in the indexed data. Selected vertical expansions offer the analysis gains to make their properties available as filters.	Horizontal expansion yields the index storage gain, but a vertical storage can sometimes be more efficient for very complex arrays that flatten horizontally to millions of columns.	Horizontal nested attributes are not well suited for filtering or aggregate analysis. Increased storage needs for the vertical expanded whitelist attributes.
Horizontal or vertical expansion with Array Flatten Depth other than Unlimited.	No expansion for any nested arrays at levels below the specified depth level. An unexpanded array and its properties are stored as a JSON string value in one column. None specifies that all arrays are indexed as native JSON strings.	Helps reduce row and/or column counts for a record when text searches within the JSON string content is sufficient for users.	JSON string values cannot support filtering or visualization operations, unless Materialize with JSONPath is used. JSON string fields can support searches, including searches with Elastic nested query syntax.
Field selection policy can be used to specify nested JSON objects (not arrays) to be indexed and stored as JSON strings.		Helps reduce the conversion of nested object properties to many rows or filtering fields that might not be very useful.	JSON string values cannot support filtering or visualization operations, unless Materialize with JSONPath is used. JSON string fields can support searches, including searches with Elastic nested query syntax.
	Materialize with JSONPath schema transformation can be used to materialize a JSON field stored inside a JSON string to create a virtual column in the view for analytics and filtering.	For important content that is stored as a JSON string, you can make that property into a virtual search filter and materialized column.
	Treat as Nested JSON schema transformation allows a JSON string column to support searches with Elastic nested query path syntax.	For important content that is stored as a JSON string, you can enable that field to support nested query path expressions to search for values with Discover or the Elastic API.