LogStash::Json::ParserError

Elastic Stack Logstash
1

Hello, I have problems with parsing json logs to logstash, there are tons of errors at log:
Error parsing json {:source=>"message", :raw=>"****", :exception=>#<LogStash::Json::ParserError: Invalid UTF-8 start byte 0xa1
I guess that I need to decode parsed json to UTF-8 charsets but I cannot find how to do it exactly. Can someone point me?

2

What does the message field look like and what is your logstash configuration?

3

sorry for long response, full error message:
Error parsing json {:source=>"message", :raw=>"Init", :exception=>#<LogStash::Json::ParserError: Invalid UTF-8 start byte 0x98
at [Source: (byte)"Ð�нÐ"; line: 1, column: 3]>}
here is input and filter part of my logstash config file:
input {
http {
host => "0.0.0.0"
port => "8002"
type => "logs"
additional_codecs => {"application/json"=>"json"}
}}

filter {

json { source => "message" }
json { source => "message" }
json { source => "message" }

mutate { remove_field => [ "host", "__rpc_corr", "rpc_node_id", "rpc_ref_id",, "headers" ] }
}

4

If you have a message field nested inside a message field nested inside a message field I suggest you add tag_on_failure options to those filters so that it is clear which one is failing.

5

I'm kinda new to configuration of elastic - can you please provide me more detailed steps how to do that?

6 
json { source => "message" tag_on_failure => [ "_jsonparsefailure", "1st" ] }
json { source => "message" tag_on_failure => [ "_jsonparsefailure", "2nd" ] }
json { source => "message" tag_on_failure => [ "_jsonparsefailure", "3rd" ] }
7

I've changed filters the way you provided but nothing changed at logs:
Error parsing json {:source=>"message", :raw=>"Ошибка {"status":"error","errorCode":"teach_session_decline","message":"Данная фукнция..."}", :exception=>#<LogStash::Json::ParserError: Invalid UTF-8 start byte 0x9e
at [Source: (byte)"��Р{"status":"error","errorCode":"teach_session_decline","message":"�анни"}"; line: 1, column: 3]>}

8

I would not expect it to. What tags are present on the event?

9

Here is full event log:
[2020-10-08T13:39:40,904][WARN ][logstash.filters.json ] Error parsing json {:source=>"message", :raw=>"Ошибка доступа{"status":"error","errorCode":"teach_session_decline","message":"Данная фукнция не доступна"}", :exception=>#<LogStash::Json::ParserError: Invalid UTF-8 start byte 0x9e
at [Source: (byte)"�� {"status":"error","errorCode":"teach_session_decline","message":"�ан"}"; line: 1, column: 3]>}
[2020-10-08T13:39:40,904][WARN ][logstash.filters.json ] Parsed JSON object/hash requires a target configuration option {:source=>"message", :raw=>""}
[2020-10-08T13:39:40,904][WARN ][logstash.filters.json ] Error parsing json {:source=>"message", :raw=>"Создана сессия", :exception=>#<LogStash::Json::ParserError: Invalid UTF-8 start byte 0xa1
at [Source: (byte)"СоÐ"; line: 1, column: 3]>}

Unfortunately I don't see any tags here

10

I'm still facing this problem. Any help would be appreciated

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.