Kibana: Field collapsing and inner hits not working for visualizations

Elastic Stack Kibana
1

Hello,

In Kibana, I have created a search that returns the last status for each servers, using field collapsing and inner hits.

{
  "query": {
    "match": {
      "event_name": "server_status"
    }
  },
  "collapse": {
    "field": "server_id",
    "inner_hits": {
      "name": "last_status",
      "size": 1,
      "sort": [
        {
          "@timestamp": "asc"
        }
      ]
    },
    "max_concurrent_group_searches": 4
  }
}

I have 13 servers in this example. When viewing the results in the Discover section, I can see 13 entries, as wanted (I created a new event for one server and should avoid having 14 entries returned).

The hits count given by Kibana at the top left is 14, but that is normal, as stated in the docs, that is the total hit count and not the inner hit.

The problem is, when creating visualisations, like a Pie Chart for instance, the total entries are taken into account (14) and not my inner hit, which should be 13.

I thus changed the Slice Size metric to a Unique Count on server_id, and I got 13 as wanted.

But as soon as a configured the Split Slices parameter on the status Term, I got back to 14 entries (the old entry for the server which had a change of status was still appearing).

How can I tell Kibana that I want my visualization to only take into account my inner hit ? Which means only taking the last status event for each server and spliting slices for each status in the Pie Chart.

If it is not possible, what would be a good alternative ?

2

Hi,

I'm sorry, but what you are asking for is not possible. What you would need is to do the aggregation on top of the collapsed documents, which Elasticsearch and Kibana can't do.

I think the best workaround you would have for now is having beside the actual index with all the data, that you currently store, another index, that only have the most recent status of each server, so basically your data delivery system, would update the documents with the latest status once a new status arrive instead of just adding them there. That way you could use that index to visualize the current state of your infrastructure and the other one to visualize historical data.

Cheers,
Tim

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.