Filed names are not renaming in logstash pipeline

Elastic Stack Logstash
1

Hi All,
I am using mutate filter to rename the fields. But only one field is renamed remaining fields are not renamed. Below the logstash configuration.

json
   {
    source => "jsonmessage"
    target => "[x][data]"
    remove_field=>["jsonmessage"]
    tag_on_failure => ["_jsonparsefailure"]
    skip_on_invalid_json =>true
   }
  mutate
   {
 	 rename => ["[x][data][loglevel]", "loglevel" ]
 	 rename => ["[x][data][request][streetName]", "[x][request][streetname]" ]
 	 rename => ["[x][data][request][operate]", "[x][request][operate]" ]
 	 rename => ["[x][data][request][bookingID]", "bookkingid" ]
 	 rename => ["[x][data][response][paymeNT]", "payment" ]
 	}

In above renaming section only first field renamed properly. remaining fields are not renamed. Similar type of configuration is working for other application logs.Please guide me on this?

2

Hello @upreddy253

Could you please share 1 logline from the log file.

Thanks!!

3

Please share some sample data and the output you are getting.

Also, can you share your configuration using the Preformatted text option? the </> button? It is not clear what that check mark icon means.

4

@leandrojmp

I formatted the text

1 Like
5

Hi @leandrojmp
Below the sample json log.

{"loglevel":"log","request":{"streetName":"abc","operate":"abc","bookingID":"xyz"},"response":{"paymeNT":"paymeNT-23290"}}

for above json message i have applied json parser and after that i have tried to rename fields. But only first field is renaming but other fields are not changed.

json
   {
    source => "jsonmessage"
    target => "[x][data]"
    remove_field=>["jsonmessage"]
    tag_on_failure => ["_jsonparsefailure"]
    skip_on_invalid_json =>true
   }
  mutate
   {
 	 rename => ["[x][data][loglevel]", "loglevel" ]
 	 rename => ["[x][data][request][streetName]", "[x][request][streetname]" ]
 	 rename => ["[x][data][request][operate]", "[x][request][operate]" ]
 	 rename => ["[x][data][request][bookingID]", "bookkingid" ]
 	 rename => ["[x][data][response][paymeNT]", "payment" ]
 	}

In kibana we are able to see only first 2 fields are renamed (loglevel,streetname) remaining fields are not renamed. remaining fields are getting like below.
x.data.request.bookingID , x.data.request.paymeNT.

@stephenb thanks for modifying query.

6

@upreddy253 please use 3 backticks ``` before and after your code so that it's readable as code it's marked down...

When you do not, your code is impossible to read

I fixed it this last time. You need to do it next time

7

What else do you have in your configuration? Can you share the full configuration? Also, what is the version of your Logstash?

It works without any issue for me, all fields are renamed.

One thing that you need to change is the syntax of the rename command, it needs to be like this.

So the correct way to format your mutate rename would be this:

    mutate {
        rename => {
            "[x][data][loglevel]" => "loglevel"
            "[x][data][request][streetName]" => "[x][request][streetname]"
            "[x][data][request][operate]" => "[x][request][operate]"
            "[x][data][request][bookingID]" => "bookkingid"
            "[x][data][response][paymeNT]" => "payment"
        }
    }