Dec 4th, 2019: [EN] Explore Elastic Common Schema (ECS) with Elasticsearch and Kibana

Introduction

Lire la version française

Many resources are already available to learn about ECS. Of course, there's the official documentation. Some users also use the CSV export, located at generated/csv/fields.csv. This CSV lets you quickly navigate and visualize all fields at once, or import the schema in a spreadsheet.

Did you know you can also import this CSV directly in Elasticsearch?

In this article, we'll import the CSV and the following dashboard:

Steps

This tutorial requires Elasticsearch and Kibana 7.2 at a minimum, but ideally 7.4 or 7.5.

Importing the ECS field definitions

Download the CSV for ECS 1.3.1, recently released. You can do so from this Github page generated/csv/fields.csv or from your terminal:

curl -O https://raw.githubusercontent.com/elastic/ecs/1.3/generated/csv/fields.csv

Next, open Kibana in the "Machine Learning" section, tab "Data Visualizer". This feature is freely available via the Basic license.

Upload the fields.csv file.

If you're using 7.4 or more recent, you can immediately click "Import" at the "File Contents" page.

If you're using 7.2 or 7.3, make sure to go in "Override Settings", select "Has header row" and apply the change. The "File stats" section should now display the right column titles. You can now click "Import".

On the next screen, "Import data", click the "Advanced" tab.

  1. Name the index "schema-explorer"
  2. Uncheck "Create index pattern"
  3. Replace the whole content of the "Mappings" section with the following:
{
  "Description": {
    "type": "keyword",
    "fields": { "text": { "type": "text" } }
  },
  "ECS_Version": {
    "type": "keyword"
  },
  "Example": {
    "type": "keyword"
  },
  "Field": {
    "type": "keyword"
  },
  "Field_Set": {
    "type": "keyword"
  },
  "Indexed": {
    "type": "boolean"
  },
  "Level": {
    "type": "keyword"
  },
  "Type": {
    "type": "keyword"
  }
}

The page should now look like:

Finally, click "Import". The field definitions are now imported in the "schema-explorer" index.

Import the dashboard

Visit this gist and download the file schema-explorer.ndjson.

In "Kibana Management" section "Saved Objects", import the file schema-explorer.ndjson.

Voilà! You can now use the "Schema Explorer" dashboard.

3 Likes