I have a JSON input containing a [message][message_json] field which is compressed & base64 encoded at the source.
This is the python code used to decode, decompress, and deserialize the field:
message['message_json'] = json.loads(zlib.decompress(base64.b64decode(message['message_json']), 15 + 32))
How can I replicate the same using logstash filter?
I tried to use logstash-filter-base64 to decode:
filter {
json {
source => "message"
target => "message_deserialized"
}
base64 {
field => "[message_deserialized][message_json]"
action => "decode"
}
But I'm getting "tags" => [[0] "_base64failure"]
error.
Update:
I tried
ruby {
init => "require 'base64'"
code => 'event.set("[message_deserialized][message_json_decoded]", Base64.decode64(event.get("[message_deserialized][message_json]")))'
}
It works. Now I need help in just decompressing the [message_deserialized][message_json_decoded]
field.
Update:
Successfully decompressed as well.
Final filter:
filter {
json {
source => "message"
target => "message_deserialized"
}
ruby {
init => "require 'base64'
require 'zlib'
require 'stringio'"
code => 'event.set("[message_deserialized][message_json_decoded]", Zlib::GzipReader.new(StringIO.new(Base64.decode64(event.get("[message_deserialized][message_json]")))).read)' }
json {
source => "[message_deserialized][message_json_decoded]"
target => "[message_deserialized][message_json_decoded_deserialized]"
}
}
system
(system)
Closed
September 1, 2021, 3:54pm
4
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.