Rename the message field from the original log

Elastic Stack Logstash
1

I have a log line.

1590644837.665697 2020-05-28 07:47:17,665 to:004915122898981|id:Hello there|status_code:SentSMSOK|status_message:abc.exmaple.com - 200|message:(2020-05-28 07:46:41) TEST: stuck-daily@example.com is OK (OK - Table:ETL_PROCESS_DAILY_LOADS HoursOld:6 CountResult:0 (10min cache))!

This is my logstash config:

input {
  stdin { }
}
filter {
        grok {
            match => [
                "message" , "%{GREEDYDATA:data}"
                ]
        }
        date {
            match => [ "timestamp" , "dd/MMM/YYYY:HH:mm:ss Z" ]
            remove_field => [ "timestamp" ]
        }


        if [data] {
           kv {
              source => "data"
              field_split => "|"
              value_split => ":"
           }
        }
}


output {
  stdout { codec => rubydebug }
}

Here is the output from logstash:

{
                                     "data" => "1590644837.665697       2020-05-28 07:47:17,665 to:004915122898981|id:Hello there|status_code:SentSMSOK|status_message:abc.exmaple.com - 200|message:(2020-05-28 07:46:41) TEST: stuck-daily@example.com is OK (OK - Table:ETL_PROCESS_DAILY_LOADS HoursOld:6 CountResult:0 (10min cache))!",
                                  "message" => "(2020-05-28 07:46:41) TEST: stuck-daily@example.com is OK (OK - Table:ETL_PROCESS_DAILY_LOADS HoursOld:6 CountResult:0 (10min cache))!",
                           "status_message" => "abc.exmaple.com - 200",
                               "@timestamp" => 2020-05-28T06:57:54.609Z,
                                 "@version" => "1",
                                     "host" => "def",
    "1590644837.665697       2020-05-28 07" => "47:17,665 to:004915122898981",
                                       "id" => "Hello there",
                              "status_code" => "SentSMSOK"
}

The message filed is being overwritten by the message field from the log file. How we can rename the message field from the log?

2

if you want to rename the message from the log then use mutate-gsub filter in the beginning of filter plugin

mutate {
	gsub => ["message", "message:", "information:"]
}

above code will replace message field as information field.

and if you want to remove the message field from output then use mutate-rename filter

  mutate {
    rename => { "message" => "information" }
  }
3

Thanks it works

1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.