Aggregate metricbeat 'host' info by ID & Unique host configuration?

Elastic Stack Elasticsearch
1

Hi there.

I'm trying to aggregate host configuration data out of metricbeat. I need to look at the documents for a single day, bucketing by ID, and return the host info. What I'm trying to do is watch for configuration changes on a daily basis.

If I use this code:

"query": {
		'bool': {
			'must': [
				{"exists": {"field": "host"}},
				{'term': {'@timestamp': date}}
			]
		}
	},
	"aggs": {
		"by_id": {
			"terms": {"field": "fields.id"}
		}
	}

I get an array out containing our fields.id numbers.

'doc_count': 1677397,  'key': '57c15917555158f06555f787'  (etc)

So far so good. But I also need to aggregate by host because that's what I'm trying to watch. The host information is a JSON object that looks like this:

"host": {
  "id": "211d31bcdb1a41c55f1455cb09b22d55",
  "containerized": false,
  "name": "servername",
  "hostname": "hostname",
  "architecture": "x86_64",
  "os": {
    "kernel": "3.10.0-1062.4.3.el7.x86_64",
    "codename": "Core",
    "platform": "centos",
    "version": "7 (Core)",
    "family": "redhat",
    "name": "CentOS Linux"
  }

I have tried nesting aggregations like this:

"aggs": {
		"by_id": {
			"terms": {"field": "fields.id"}
		},
		"aggs": {
			"by_host_config": {
				"terms": {"field": "host"}
			}
		}
	}

But I get an error that says "unable to parse BaseAggregationBuilder with name [by_host_config]: parser not found" even though all my books and all of the documentation I can find shows doing this kind of aggregation just like this...

What am I doing wrong? Ugh!

Thank you!

2

Just to make my clear as I can, this is the kind of output I need:

{
    'doc_count': 16876,  
    'id': '57c15917555158f06555f787'
    'host': {
        ...
        "hostname": "hostname",
        "architecture": "x86_64",
        "os": {
             "kernel": "3.10.0-1062.4.3.el7.x86_64"
             ...
         }
    }
},
{
        'doc_count': 16556,  
        'id': '59c15917555199f06555ju87'
        'host': {
        ...
        "hostname": "hostname2",
        "architecture": "x86_64",
        "os": {
             "kernel": "3.10.2-1066.4.3.el7.x86_64"
             ...
         }
    }
},
...

Thank you again for any input you may have. : )

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.