Inject extra fields into metricbeat stats

[ld_pvl]

Hi Elastic Team,

I've been reading about creating a beat and creating metricsets for metricbeat but would like your advise on creating extra fields.

What I would like to do is to add one extra field such as datacenter_name but in all available metricsets in the metric module system - would I have to download the metricbeat source code and amend each individual metricsets?

Cheers,

[andrewkroh]

You can add this data to all events generated by using fields.

fields_under_root: true
fields:
  datacenter_name: mydc

If you are deploying in a cloud environment you could use our add_cloud_metadata processor to automatically determine this info and add it.

processors:
- add_cloud_metadata: ~

For example on AWS this would add this type of info to each event:

{
  "meta": {
    "cloud": {
      "availability_zone": "us-east-1c",
      "instance_id": "i-4e123456",
      "machine_type": "t2.medium",
      "provider": "ec2",
      "region": "us-east-1"
    }
  }
}

[ld_pvl]

We're not deploying in a cloud environment unfortunately.

Is there a way to do this programmatically/dynamically? I have a way to extract the data center name based on the host where metricbeat is installed/running and I want to inject that dynamic value into an extra field into all the metricsets - using fields it looks like requires hard coding in metricbeat.yml if I'm not wrong.

[andrewkroh]

The only way I can think of would be to inject the data into the process's environment then reference the environment var from the config. https://www.elastic.co/guide/en/beats/metricbeat/5.5/using-environ-vars.html

[ld_pvl]

That sounds like it could be a simple solution - I'll report back with my findings.