Add Elastic Agent tags to log events

[DougR]

Is there a way to append the Elastic Agent tags to the event, or at least to log events? My use case is that I designate agents by application environment (e.g., dev/qa/stage/prod), and would like that tag to be available in the actual event.

Thx.

Update

An alternative to this would be to add a tag to each event generated by an agent. E.g., with the add_tags or add_labels processor. I can do this with any custom log and with many of the built-in logs (although not all), and I can't do this with most of the metrics.

Thx.

[BenB196]

A workaround I use for this today is using the data_stream.namespace value to designate an environment. It's not 100% ideal in all use-cases as it could increase the number of overall indexes as you'll break them out by environment.

There is also this open issue ([Elastic Agent] Support Global processors on a per policy basis · Issue #29459 · elastic/beats · GitHub) which in theory would allow for global processors at which point it'd be fairly easy to add tags/labels/additional fields.

[DougR]

@BenB196

A workaround I use for this today is using the data_stream.namespace value to designate an environment. It's not 100% ideal in all use-cases as it could increase the number of overall indexes as you'll break them out by environment.

That's the workaround that I was looking at as well. I was just hoping for a different solution.

[BenB196]

Yeah, unfortunately I ran into the same issue you did where not all integrations support setting processors.

There are 2 other solutions, but I dislike them as they cause additional complexities.

If you know what subnets belong to which environments, you could:

1. Output the logs to Logstash and have it add the data by parsing the Agent Host IP and setting the field via subnet.
2. Similarly you can use Elasticsearch ingest pipelines with the set processor (using script filter) or enrich filter (if you have IP ranges/environments as an index)
   • This solution is kind of annoying today, but when 8.4 comes out it would be slightly easier as you can add custom pipelines to all integrations.

[BenB196]

Btw, side note, I'm not sure if you've looked into it, but ECS has the service.environment field (Service Fields | Elastic Common Schema (ECS) Reference [8.3] | Elastic) which I leverage to keep environment field consistent across different things.

[DougR]

@BenB196

Btw, side note, I'm not sure if you've looked into it, but ECS has the service.environment field (Service Fields | Elastic Common Schema (ECS) Reference [8.3] | Elastic) which I leverage to keep environment field consistent across different things.

I saw this yesterday. It's a new(ish) field, looks like it's still part of the beta specification. I've used labels.env in the past, but will be transitioning.

@BenB196

If you know what subnets belong to which environments, you could:

1. Output the logs to Logstash and have it add the data by parsing the Agent Host IP and setting the field via subnet.
2. Similarly you can use Elasticsearch ingest pipelines with the set processor (using script filter) or enrich filter (if you have IP ranges/environments as an index)

• This solution is kind of annoying today, but when 8.4 comes out it would be slightly easier as you can add custom pipelines to all integrations.

I thought of this as well, but I've done it before and I totally agree that it's messy. I'm absolutely not going to take this route. I'll use workarounds between now and then.

I see that the global processor on a per-policy basis appears to have been slated for 8.5.0, and 8.4.0 was just released today, so hopefully it won't be that long to wait.

Thx!

[BenB196]

Yep, no problem.

I see that the global processor on a per-policy basis appears to have been slated for 8.5.0, and 8.4.0 was just released today, so hopefully it won't be that long to wait.

I believe that in 8.5.0 they'll be release v2 as experimental (from the issues I've seen about it), might need to wait for 8.6.0 to it to be on-by-default.

Though someone from Elastic would probably need to confirm this, as I'm just making assumptions from some issues I read while looking into this issue.

[DougR]

It just occurred to me - are the tags from the elastic-agent sent as part of the event metadata at all? If so, can I look for them in the metadata?

Thx.