Step 4. Search and Visualize your Indexed Data

Use Analytics and the Kibana interface to search and analyze your log and event data.

ChaosSearch includes an embedded Kibana interface that you can use to query and visualize the indexed data for your log and event content. After you create one or more views for your ChaosSearch object groups, go to the Analytics area to search and visualize your data.

👍

More Information

Kibana has many features for searching within and visualizing the data in views, and for creating all-in-one dashboards that enable side-by-side evaluations of data. See Kibana for a detailed description about using the Kibana/Analytics area, including search options and best practices, creating visualizations, and creating dashboards.

In the ChaosSearch console, click Analytics to open Kibana. The Discover page is a common starting point to run a basic search of your indexed data. There are five steps to run a basic search on your data:

  1. Make sure that you are on the Discover page.

  2. Select a Refinery view in the drop-down list to specify the indexed data that you want to search. (Refinery views are the ChaosSearch equivalent to Kibana index patterns.)

  3. Specify one or more search terms in the Search field. (If you leave Search blank, as in this example, the search returns all of the records for the time frame, which is not a good practice when your indexed data has millions or billions of records.)

  4. Specify a time frame. The default is the last 15 minutes of data. For live indexes, the last 15 minutes is usually a good sample size. For static indexes, the events and log data could have occurred much earlier than the last 15 minutes, so you might need to adjust the time range to one more aligned with the timestamps of the entries in the indexed data (as in this example).

  5. Click Refresh (or Refresh data) to run the search.

In this example, the search field was left empty to return all records, which might be reasonable for a small data set like this, which is a small orders table based on TPC-H data sets. ChaosSearch is designed to index data at scale including complex application logs, which means that an unbounded search for a longer time range could return billions of records. A free text search with no terms, or very common terms, could take a long time to return a set of results that could be too large to scan easily.

👍

NOTES:

Kibana supports many search value options such as wildcards, field-level searches, filter searches and combinations. More information is available in the later Kibana help topics.

The Discover page uses Kibana Query Language (KQL) by default. If you click the KQL link, you can turn off KQL and use Lucene search syntax instead.

As an example of a more bounded search using the orders data, you might want to see records that reference a specific clerk ID. If you know which field (o_clerk) contains the information that you are searching for, field-level searches return a more granular set of results, and in less time. For example, to find the orders processed by Clerk#000000497, type o_clerk:Clerk#000000497 in the search window and refresh. Search values could be case-sensitive or case-insensitive, depending upon how the Refinery view was created.

For this sample data, five results are returned.

You can combine the field-level search criteria with AND, OR, and NOT syntax to create even more granular searches.

Visualizations and Dashboards

Some users might have pre-built dashboards or visualizations that offer graphical or tabular representations of data analysis for their log and event files, built by ChaosSearch Customer Success or customer data analysts.

Click the Visualize or the Dashboard options in the left menu to see if there are pre-built visualizations that you can use for your data. The following image is a sample visualization of the orders data showing a trend of orders by priority:

Visualizations offer another representation to turning your search queries into graphical or tabular displays that can be quickly scanned to show important information in your indexed data.

Dashboards combine multiple visualizations (either saved visualizations or ad-hoc ones created during dashboard development) on one page so that users can compare important factors for the data in a side-by-side summary. A sample dashboard for orders data follows with the bar chart visualization and another pie chart visualization:

Creating visualizations and dashboards can take some time and practice for new users to learn, especially for developing the analytics that support them. The Kibana interface offers guidance to help with the process for creating them.


Did this page help you?