Event.remove not working

[Akash_Kumar_Dutta]

I am using the following ruby filter to update the fields.

def filter(event)
  keys = event.to_hash.keys
  keys.each{|key|

    if ( key.start_with? '_' and key != "_@timestamp")
      value = event.get(key)
      newkey = key.sub!(/^_/, '')
      event.remove(key)
      event.set(newkey , value)

    elsif (key == "_@timestamp")
      value = event.get(key)
      event.set("@apptimestamp", Time.at(value))
      event.remove(key)
    end
      
  }
  return [event]

  rescue Exception => e
    event.set('logstash_ruby_exception', 'underscores: ' + e.message)
    return [event]
end

The code duplicates fields starting with underscore, to fields not starting with underscore. But for some reason doesn't eliminate the one's starting with underscore. That is, event.remove doesn't work as expected. Any ideas?

Thanks in advance!

[guyboertje]

What version of Logstash?

[Akash_Kumar_Dutta]

6.2.4
image used is docker.elastic.co/logstash/logstash:6.2.4

[guyboertje]

@Akash_Kumar_Dutta

newkey = key.sub!(/^_/, '')

sub! alters the value of key so it is not removable.
In Ruby IRB...

irb(main):001:0> key = "_abc"
=> "_abc"
irb(main):002:0> key.sub!(/^_/, "")
=> "abc"
irb(main):003:0> key
=> "abc"

Looking at how the mutate filter works and with my knowledge of ruby I suggest this improvement.

input {
  generator {
    message => '{"_a": "A", "_b": "B", "c_": "C"}'
    count => 1
  }
}

filter {
  json {
    source => "message"
  }
  if "_jsonparsefailure" not in [tags]  {
    ruby {
      code => '
        keys = event.to_hash.keys
        keys.select{|key| key.start_with?("_") && key != "_@timestamp" }.each do |key|
          newkey = key[1, key.length] # we know the first character is an underscore
          event.set(newkey, event.remove(key))
        end
        # deal with _@timestamp here, not inside the iterator
      '
    }
  }
}

output {
  stdout {
    codec => rubydebug
  }
}

Gives

{
      "sequence" => 0,
             "a" => "A",
             "b" => "B",
    "@timestamp" => 2018-06-01T09:57:42.934Z,
      "@version" => "1",
          "host" => "Elastics-MacBook-Pro.local",
       "message" => "{\"_a\": \"A\", \"_b\": \"B\", \"c_\": \"C\"}",
            "c_" => "C"
}

[Akash_Kumar_Dutta]

Thank you so much! I am so silly, didn't notice that!