Filter json parse is replacing tags set from the input

Elastic Stack Logstash
1

I have input that sets tags

input {
  ...
  tags => ["a"]
}

But in the filter, I do a json parse:

filter {
  json {
    source => "i"
  }
}

And part of that parse, includes tags. However, the tags it has replace the tags I set in the input. How can I make it append to the tags instead of replacing them?

I'm thinking I need to copy my tags to a temporary variable and then append them back into the event tag?

2

I think so too.

    mutate { copy => { "tags" => "[@metadata][inputTags]" } }
    json { source => "message" remove_field => [ "message" ] }
    ruby { code => 'event.get("[@metadata][inputTags]").to_a.map { |x| event.tag(x) }' }

Note the .to_a in the last line, if [@metadata][inputTags] is nil then I think this creates an empty array, so calling .map on it is a no-op.

3

I think that works.

I was trying to do it through mutate, copy and add_tag, it just kept converting it to a string with comma.

mutate {
  add_tag => [ "%{tmp_tags}" ]
}

     "tags" => [
        [0] "aabc",
        [1] "def",
        [2] "a,b"
    ]

I tried merge, seems to work too:

merge => { "[tags]" => "[@metadata][inputTags]" }

One side effect is if the source doesn't have any tags, it creates duplicates. The event.tag(x) seem to have some duplication protection.

   "tags" => [
        [0] "abc",
        [1] "def",
        [2] "abc",
        [3] "def"
    ]
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.