Issues with Gsub moving from Logstash 7 to 8

Elastic Stack Logstash
1

Hi all, as the title says, I'm experiencing some issues with GSUB, moving from logstash 7 to logstash 8. I discarded a problem with ECS compatibility (I didn't think it was involved here, but just in case), so it must be due to the changes in Java or jruby, I don't know because I can't find any documentation.

The situation is as follows, I have a pipeline where I send raw logs that are then processed as json, but before turning them into JSON, I have to obfuscate all "username;password" messages.

mutate {
		gsub => [ "message", '(?<=Password[\=])(.*(?=;)|.+?(?=\"))', "##REDACTED##" ]
	}

And the part of the message it is supposed to obfuscate looks like this:

"message": "Whatever=Dummy;Password=Kitty"

as it is part of a raw log, before being parsed as json, it will look something like:

\"message\": \"Whatever=Dummy;Password=Kitty\",

This does work in logstash 7, and all the regex testers match it perfectly.

Any idea of what could be wrong?

3 
input { generator { count => 1 lines => [ '{"message": "Whatever=Dummy;Password=Kitty"}' ] } }
output { stdout { codec => rubydebug { metadata => false } } }
filter {
    mutate { gsub => [ "message", '(?<=Password[\=])(.*(?=;)|.+?(?=\"))', "##REDACTED##" ] }
}

works in 8.16. Can you provide a reproducible example that does not work?

1 Like
4

Thank you Badger, thanks to you I found the problem. Turns out the event.original which is the one that returned a different value, was populated from the beginning, so I just had to overwrite it after doing the gsub.