Remove duplicate field value

In my logstash output I received following output.

"log_name" => [
    [0] "Authentication",
    [1] "Authentication"
               "log_level" => "info",
              "session_id" => [
    [0] "4280340685",
    [1] "4280340685"
            "header_flags" => [
    [0] "Encrypted",
    [1] "Encrypted"

In that output, for filed name log_name, session_id and header_flags have duplicate filed values.

How do I remove those duplicate field value under logstash filter, without using KV

1 Like

What does your config look like? Do you be any chance have multiple grok patterns that can match the same event and therefore extract the same field twice?


First message match a grok pattern and then followed by KV pair

Can you show the config and a sample raw message?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.