How to optimise/cleanup config with multiple mutations?

[GitSpree23 (LogSpree)]

There are a lot of new field creations, renames, and removals in my filter plugin. And if, for example, I try to club the renames together, the mutations don't get executed correctly and i end up with only half the mutations applied.

This is my filter plugin:

filter {
  json {
    source => "message"
    target => "message_deserialized"
  }

  ruby {
    init => "require 'base64'
             require 'zlib'
             require 'stringio'"
    code => 'event.set("[message_deserialized][message_json_decoded]", Zlib::GzipReader.new(StringIO.new(Base64.decode64(event.get("[message_deserialized][message_json]")))).read)' }

  json {
    source => "[message_deserialized][message_json_decoded]"
    target => "[message_deserialized][message_json_decoded_deserialized]"
  }

  mutate {
    remove_field => [ "message", "[message_deserialized][message_json]", "[message_deserialized][message_json_decoded]" ]
  }

  mutate {
    rename => { "[message_deserialized][message_json_decoded_deserialized]" => "[message_deserialized][message_json]" }
  }

  mutate {
    rename => { "message_deserialized" => "message" }
  }

  split {
    field => "[message][message_json]"
  }
}

How to reduce LoC and optimise the transformations to execute faster?

[Badger]

@GitSpree23

And if, for example, I try to club the renames together, the mutations don't get executed correctly and i end up with only half the mutations applied.

You have to use multiple mutate filters if you need to constrain the order. That applies to both the order in which options execute (see here and here), as well as the order of entries in an option.

If you do

mutate {
    rename {
        "[foo][bar]" => "a"
        "[foo]" => "b"
   }
}

does the entry within [foo] get renamed before [foo] itself moves? The answer is that it used to, but in (I think) 7.12 and newer the order is (sometimes?) switched, so that when it tries to rename [foo][bar] it no longer exists.

I cannot conceive that the performance impact of having multiple filter will be significant.