Rollup groups vs terms - how to filter and group by keyword terms?

Elastic Stack Kibana
1

Perhaps I'm not understanding how terms work with rollups.

Attempting to summarize metricbeat details into hourly rollup indexes, but when using the rollup index on a Lens the value of host.hostname.terms.value is just couple of GUID values (not all of the various server hostname values where metrics have been gathered from).

Was expecting to have rollup index containing calculated min, max, avg values available to be grouped by the terms.

What exactly does specifying Terms do? Thought that allowed for filtering as well as grouping on those terms, but clearly that isn't possible since its just showing a GUID.

Here's the test rollup job definition...

PUT _rollup/job/rollup-metricbeat-202205
{
      "index_pattern": "metricbeat-7.16.3-2022.05.01",
      "rollup_index": "rollup-metricbeat-202205",
      "cron": "0 1 * * * ?",
      "page_size": 1000,
      "timeout": "600s",
      "groups": {
        "date_histogram": {
          "fixed_interval": "60m",
          "field": "@timestamp",
          "delay": "1h",
          "time_zone": "UTC"
        },
        "terms": {
          "fields": [ "windows.service.name", "host.hostname", "windows.service.state", "mssql.database.name" ]
        }
        },
        "metrics": [
        {
          "field": "system.cpu.idle.pct",
          "metrics": [
            "avg",
            "max",
            "min"
          ]
        },
:snip:
2

Your config looks good - I'm able to get a group by on a field set up on the "terms" in Lens for a min/max metric. Can you check that your data view a rollup data view (it's a separate type of data view in Kibana--you specify it when you create your data view / index pattern)

Try not using an average & a breakdown as this throws an error. Min/Max should work. This is a known issue and something we're working on via a new version of rollup. Check out Phase 3.3

Here's an example of my rollup vis working as you describe with a terms field

image
image2376×1514 313 KB

3

"Can you check that your data view a rollup data view (it's a separate type of data view in Kibana--you specify it when you create your data view / index pattern)"

When I use Discover to query the rollup index, I'm seeing a GUID value for the host.hostname.terms.value and other terms defined.

Creating a Rollup type of Kibana index pattern didn't do anything different (the data stored in the rollup index to group on specific hostname values doesn't seem to be there... it's showing only a GUID value e.g. "66df99c4c3224f73bbb9baa940418ab4-2196702451" instead of "servernameA", "servernameB", etc).

Seems like the rollup job isn't keeping separate terms to allow grouping on anything meaningful.

4

What version are you on?

5

What version are you on?

7.16.1 currently.

6

Any chance you can upgrade to latest? I'm not seeing this behavior on my end

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.