How to group documents and show latest document per group

Elastic Stack Kibana
1

Scenario: We have a third party tool for server patching and it sucks when it comes to reporting. So I want to build a dashboard in Kibana which shows patch status of thousands of servers.

Available data so far per server: Hostname, Operation System, scan run identifier, scan run date, list of missing patches, number of missing patches

Scans run in different batches across all servers, so there are multiple scans which show the current status. You have to combine these runs to see the whole current picture for all servers.

One thing I want to display in Kibana: Number of missing patches in last scan run across all servers. The number of servers is dynamic and not fix, because some new VMs are created and some old ones destroyed every day

Question: Should I try to create buckets for each server with a formula and function „terms“ by using „hostname“? This will lead to many thousands of buckets over time and I‘m concerned about performance. Or can I also get the latest document per server in another way without „terms“ function? The hard thing is that the scans run unpredictable and not at fixed times per day.

Any thoughts how to work with this?

2

Have you checked Transforms and the "Latest Transforms" variant?

With transforms, you can automatically generate a side index that contains only the last document for each server and, with that, perform any aggregations and reporting you need.

3

That sounds very promising. I reach out to our platform admins if they have any concerns with it.
Thanks a lot for this hint

In case they have objections.. do you have another idea how to tackle this problem?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.