New tag applied to every document?

Elastic Stack Logstash
1

I've updated everything to the latest versions (logstash, elasticsearch, filebeat, kibana) and I see this tag added to every document: beats_input_codec_plain_applied

Everything looks to be working but what does this mean and do I care?

1 Like
2

I also recently upgraded to the latest versions and see this "tags" : [ "beats_input_codec_plain_applied" ] in every document.

Is there a setting to turn this off?

3

I've updated one of my beats input servers and also noticed this tag has been added to each of the requests. Curious regarding it's meaningfulness, or else I may filter it out.

4

I'm seeing the same thing. It seems rather unnecessary, and TBH is making a mess of my tags.

I don't see anything in the docs about disabling it.

[edit] This is in the beats input code. I guess this should be moved to the Beats category?

5

A couple of devs from team went Elasticon and asked about this. Seems like this tag is here to stay and you will need to manually remove it.

6

What's the purpose for it? Is it documented anywhere?

7

Thinking more about this I guess it could be useful info to have in the LS pipeline, but it doesn't really need to need to exist for the lifetime of the message. Removing them manually if you don't want them is just extra processing.

Would the Beats team consider moving these tags to metadata instead?

2 Likes
8

and i've just updated to logstash 2.2 with the latest logstash-input-beats plugin just to discover that all tags are unusable ...

9

Bumping this again as this compulsory tag is causing real problems.

This tag turns up in every single document we ship. Which means it also shows up in any vis that includes tags. We can't even exclude this tag from the vis, because it's in every document, so all documents would be excluded. Because it's in every document, the tags we're searching for are often buried under an avalanche of results for the beats_input_codec_plain_applied tag.

We can't reliably strip the tag after the input, as some shippers are setting custom tags so we can't assume it's at position 0 in the tag array. Our only options seem to be:

  • never use tags at the shipper end (and strip all tags after arrival)
  • put up with this tag showing up in all our vis panels

TBH neither of these are very appealing.

Can someone look into this please?

10

Bumping this again... I'm currently on nearly the latest 2.x versions and preparing to update to 5.2 and am wondering if this has been "fixed" in the 5.x release?