I am trying to parse json coming over a tcp port. I noticed json parse errors and it looks like its due to some non-json strings in the input stream. I then crafted a regexp that matches these spurious messages and tried to gsub it with "" string so that only pure json is left behind. Now, I am seeing this error:
[DEBUG] 2019-06-06 18:29:19.472 [nioEventLoopGroup-2-1] ResourceLeakDetectorFactory - Loaded default ResourceLeakDetector: io.netty.util.ResourceLeakDetector@44f418a3
[ERROR] 2019-06-06 18:29:19.617 [nioEventLoopGroup-2-1] tcp - Error in Netty pipeline: org.jruby.exceptions.EncodingError$CompatibilityError: (CompatibilityError) incompatible character encodings: ASCII-8BIT and UTF-8
I tried to substitute the char encoding as shown below but it does not help:
input {
tcp {
port => 48203
codec => json_lines { charset => "UTF-8"
delimiter => "\\\w+[0-9.'ɰ]?" }
}
}
filter {
mutate {
gsub => ["fieldname", "\\\w+[0-9.'ɰ]?", "" ]
}
}