Rename field problem

[ddoroshenko (Dmitriy)]

Hello!

I have input data like

"abc":"123";"def":"456";"ghi":"789"

I use the following filter to parse data and rename "abc" field to "blabla"

filter {
  kv {
    source => "message"
    field_split => ";"
    value_split => ":"
    trim_key => " "
  }
  
  mutate {
    rename => { "abc" => "blabla" }
  }
}

Data are parsed but the field "abc" is not renamed.

How to solve the problem?

Note: I can rename the "message" field with no problem, but not "abc"

[Alex_Marquardt]

I suspect that you are storing field names that are getting extracted including quotation characters in the field name. You could remove them with remove_char_key.

The pipeline below demonstrates how to get the rename to work for such a scenario:

input {
# The generator creates an input event
    generator {
        lines => '"abc":"123";"def":"456";"ghi":"789"'
        count => 1
    }
}
filter {
    kv {
      source => "message"
      field_split => ";"
      value_split => ":"
      trim_key => " "
      remove_char_key => "\""
    }

    mutate {
      rename => {"abc" => "blabla"}
    }
}

output {
    stdout { codec => "rubydebug" }
}

The output from the above pipeline is the following, which shows that the field has been renamed correctly:

{
        "blabla" => "123",
          "host" => "New2020MacBook",
    "@timestamp" => 2021-11-19T13:48:09.557Z,
           "def" => "456",
      "sequence" => 0,
           "ghi" => "789",
      "@version" => "1",
       "message" => "\"abc\":\"123\";\"def\":\"456\";\"ghi\":\"789\""
}

Without remove_char_key the output would look as follows (which is unlikely what you want):

{
       "\"abc\"" => "123",
          "host" => "New2020MacBook",
       "\"def\"" => "456",
       "\"ghi\"" => "789",
    "@timestamp" => 2021-11-19T13:45:12.219Z,
      "sequence" => 0,
      "@version" => "1",
       "message" => "\"abc\":\"123\";\"def\":\"456\";\"ghi\":\"789\""
}

[ddoroshenko (Dmitriy)]

@Alex_Marquardt thank you! You helped me a lot!