Data Table Visualizations - Newbie wrestling with Null Values & Counts

Elastic Stack Kibana
1

I am new to Kibana (v6.86). I have been wrestling to create some visualizations and dashboards over the past few weeks. I am not very technical... to date, pretty much all of my work has involved the selectors that exist in the various visualizations and not actual coding. I wonder if someone might provide some plain-english insight into two areas when working with DATA TABLE visualizations:

  1. NULL VALUES
    I understand that Kibana visualizations are based on data that actually exists, but there are circumstances in which I want to account for non-existent data. In particular, I am trying to get a "Sibling Pipeline | Average Bucket" aggregations to show average activity over time.
    Logically, one would take the total activity, and divide it over the timeframe: 100 flights over 10 days yields an average of 10 flights per day.
    But Kibana doesn't account for null records. So: Zero flights on 9 days and 100 flights on one day yields an average of 100 flights per day!
    I gather that there is a JSON property {"min_doc_count":0} that seems like it might address this issue, but I've been unsuccessful at integrating that code into a Sibling Pipeline | Average Bucket Aggregation.

  2. COUNT
    I gather that COUNT is a special field, and wonder how I can make more use of it. Specifically, I'd like to establish a "Sibling Pipeline Aggregation" to report the Standard Deviation of COUNT Per Day. Can this be done?

1 Like
2

Any thoughts, anyone? Even if I need to learn to do some (rudimentary) coding... Or does Kibana not lend itself to working with small result sets in this manner? I'm at a loss - any insight would be appreciated!

Thank you!

3

Regarding #1: You're right in your assessment that Kibana & Elasticsearch are meant for working with data that exists. If you really want to account for buckets where data does not exist, you'll probably want to bucket using the filters aggregation and specifically create a filter matching the bucket that has no data.

Regarding #2: This can't currently be done in Kibana today, but it could (and should) definitely be added. I think the best place to keep track of progress towards it (and other currently unsupported Elasticsearch aggregations) is here:

4

Thank you very much Lukas.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.