Filter for fields that are present in Data Views

Elastic Stack Kibana
1

When our users are going through the various data sources present in our Elastic stack, they often struggle to know which fields are present in order to build dashboards and alerts.

For example: For Metricbeat data, the Discover app will present all the possible fields that can be present based on the loaded templates. This list is obviously much longer than the actual present fields.

Is there a way to generate a list (either in Kibana or outside using APIs) that will show which fields are actually populated with data?

2

Hello Bjorn,

I think the best way is to copy the used Index Template and then customize the copy to only affect your index, increase priority (so your custom template will be used instead of the default one) and only add the fields you require (you may need to remove one of the component templates that contain the unnecessary mappings).

Best regards
Wolfram

3

Hi @DataBjorn,

Welcome to the community!

Fields capabilities API Field capabilities API | Elasticsearch Guide [8.14] | Elastic allows to filter out empty fields when include_empty_fields=false is specified. Example:

POST kibana*/_field_caps?include_empty_fields=false
{
  "fields": ["*"]
}
4

There is also drop_null_columns for ES|QL queries ES|QL query API | Elasticsearch Guide [8.14] | Elastic

5

Thanks - I will certainly have a look at that!