Remove duplicate field value

Elastic Stack Logstash
1

In my logstash output I received following output.

"log_name" => [
    [0] "Authentication",
    [1] "Authentication"
],
               "log_level" => "info",
              "session_id" => [
    [0] "4280340685",
    [1] "4280340685"
],
            "header_flags" => [
    [0] "Encrypted",
    [1] "Encrypted"
],

In that output, for filed name log_name, session_id and header_flags have duplicate filed values.

How do I remove those duplicate field value under logstash filter, without using KV

1 Like
2

What does your config look like? Do you be any chance have multiple grok patterns that can match the same event and therefore extract the same field twice?

3

Yes,

First message match a grok pattern and then followed by KV pair

4

Can you show the config and a sample raw message?

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.