Input tcp codec fluent: invalid byte sequence in UTF-8 in regex

Elastic Stack Logstash
1

Hi folks,

I'm using input tcp codec fluent and getting error " invalid byte sequence in UTF-8" when doing regex. It seems there is an invalid character in the message.

Any idea how to solve it?

Is there a way to simply drop lines with this error?

Thanks

Pipeline

input {
      tcp {
        codec => fluent
        port => XXXX
      }
}

filter {

    if [message] == "" or [message] =~ /^at / {

        drop {}

    } else if [kubernetes][namespace_name] !~ /^(tst|hml)-psg-financeiro-iris/ {

         drop {}

    }

      json { source => "message" }

      if "_jsonparsefailure" in [tags] {
       drop { }
     }

}

output {
   if [kubernetes][namespace_name] =~ /^tst-psg-financeiro-iris/{
       elasticsearch {
               hosts => ["https://lixxxx.portoseguro.brasil:xxxx"]
               index => "openshift.tst-psg-fin-iris-%{+yyyy.MM.dd}"
               #document_type => "doc"
               user => "xxxxx"
               password => "xxxxxx"
               ssl => true
               ssl_certificate_verification => true
               truststore => "/elasticsearch/elasticsearch/config/certs/elastic-corphml.keystore"
               truststore_password => xxxxxx
      }
   }

   if [kubernetes][namespace_name] =~ /^hml-psg-financeiro-iris/{
       elasticsearch {
               hosts => ["https://xxxxx.portoseguro.brasil:xxxxx"]
               index => "openshift.hml-psg-fin-iris-%{+yyyy.MM.dd}"
               #document_type => "doc"
               user => "xxxxxxx"
               password => "xxxxxxxx"
               ssl => true
               ssl_certificate_verification => true
               truststore => "/elasticsearch/elasticsearch/config/certs/elastic-corphml.keystore"
               truststore_password => xxxxxx
      }
   }
}

The error:
[2022-02-03T17:37:57,298][ERROR][logstash.javapipeline ][pipeline_iris] Pipeline worker error, the pipeline will be stopped {:pipeline_id=>"pipeline_iris", :error=>"(ArgumentError) invalid byte sequence in UTF-8", :exception=>Java::OrgJrubyExceptions::ArgumentError, :backtrace=>["org.jruby.RubyRegexp.match?(org/jruby/RubyRegexp.java:1178)", "Elasticsearch.logstash_minus_7_dot_13_dot_1.logstash_minus_core.lib.logstash.java_pipeline.start_workers(/Elasticsearch/logstash-7.13.1/logstash-core/lib/logstash/java_pipeline.rb:295)"], :thread=>"#<Thread:0x1f7b1735 sleep>"}

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.