Issue with logstah and caracter encoding

Elastic Stack Logstash
1

Hello, I have a logstash input:

input {
  syslog {
    host => "127.0.0.1"
    port => 10515
    type => "nats"
  }
}

it's comming from a custom app. If I tcpdump the udp traffic on that port, here is what I get in payload

T XXXXXXX:47218 -> XXXXXXX:41234 [A] #46
..json........^..W......message..{"date":"12:15:19 GMT+0000 (Coordinated Universal Time)","microtime":1588248919449,"message":null,"type":"sdpcaller","server_addr":"172.20.1.23","server_id":"sdpcaller2.dev","subject":"INFO : Nats : Open Connection and Subscription","level":6}...^..W.|....message...{"date":"12:15:19 GMT+0000 (Coordinated Universal Time)","microtime":1588248919473,"message":null,"type":"sdpcaller","server_addr":"172.20.1.23","server_id":"sdpcaller2.dev","subject":"INFO : Nats : Stan Client : Nats Client : Servers","level":6,"misc":{"server_id":"NC5G7CBHPMLQWHXR6LBXXXXXXXXXXXGXSNFEZ2ERNAI","version":"2.0.4","proto":1,"git_commit":"c8ca58e","go":"go1.11.13","host":"nats-03","port":4222,"max_payload":1048576,"client_id":8,"connect_urls":["nats-03:4222","nats-02:4222","nats-01:4222"]}}...^..W......message...{"date":"12:15:19 GMT+0000 (Coordinated Universal Time)","microtime":1588248919481,"message":null,"type":"sdpcaller","server_addr":"172.20.1.23","server_id":"sdpcaller2.dev","subject":"INFO : Nats : Stan Client : Connect","level":6,"misc":{"server_id":"NC5G7CBHPMLQWHXR6LBDMBXFSG3JXXXXXXXEPA5Y5GXSNFEZ2ERNAI","version":"2.0.4","proto":1,"git_commit":"c8ca58e","go":"go1.11.13","host":"nats-03","port":4222,"max_payload":1048576,"client_id":8,"connect_urls":["nats-03:4222","nats-02:4222","nats-01:4222"]}}...^..`...G..message..{"date":"12:15:28 GMT+0000 (Coordinated Universal Time)","microtime":1588248

T XXXXXX:47218 -> XXXXXX1:41234 [A] #48
928137,"message":null,"type":"maestrocaller","server_addr":"172.20.1.24","server_id":"maestrocaller.dev","subject":"INFO : Nats : Open Connection and Subscription","level":6}...^..`.NB...message...{"date":"12:15:28 GMT+0000 (Coordinated Universal Time)","microtime":1588248928152,"message":null,"type":"maestrocaller","server_addr":"172.20.1.24","server_id":"maestrocaller.dev","subject":"INFO : Nats : Stan Client : Nats Client : Servers","level":6,"misc":{"server_id":"NC5G7CBHPMLQWHXR6LBDMXXXXXXGXSNFEZ2ERNAI","version":"2.0.4","proto":1,"git_commit":"c8ca58e","go":"go1.11.13","host":"nats-03","port":4222,"max_payload":1048576,"client_id":9,"connect_urls":["nats-03:4222","nats-02:4222","nats-01:4222"]}}...^..`..[...message...{"date":"12:15:28 GMT+0000 (Coordinated Universal Time)","microtime":1588248928159,"message":null,"type":"maestrocaller","server_addr":"172.20.1.24","server_id":"maestrocaller.dev","subject":"INFO : Nats : Stan Client : Connect","level":6,"misc":{"server_id":"NC5G7CBHPMLQWHXR6LBDMBXFSG3J2H2ZGSUIPEPA5Y5GXSNFEZ2ERNAI","version":"2.0.4","proto":1,"git_commit":"c8ca58e","go":"go1.11.13","host":"nats-03","port":4222,"max_payload":1048576,"client_id":9,"connect_urls":["nats-03:4222","nats-02:4222","nats-01:4222"]}}...^..`..`...message...{"date":"12:15:28 GMT+0000 (Coordinated Universal Time)","microtime":1588248928450,"message":null,"type":"devastator","server_addr":"172.20.1.19","server_id":"deva

T XXXXX:47218 -> XXXXX:41234 [AP] #50
stator-01.dev","subject":"INFO : Regenerating configuration...","level":6,"error":"No config found in redis"}...^..k..7...message...{"date":"12:15:39 GMT+0000 (Coordinated Universal Time)","microtime":1588248939308,"message":null,"type":"devastator","server_addr":"172.20.1.19","server_id":"devastator-01.dev","subject":"INFO : Regenerating configuration...","level":6,"error":"No config found in redis"}

You can see no special characters or non UTF-8 but logstash complains that:

Apr 30 09:42:19 XXXX.XXXX.XXXXX.com logstash[14864]: [2020-04-30T09:42:19,480][WARN ][logstash.codecs.jsonlines][main] JSON parse error, original data now in message field {:error=>#<LogStash::Json::ParserError: Unexpected character ('\' (code 92)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')

and this kind of long escaped string :
1\\vxī\\x81\\xA7message\\xC5\\u0001\\u0010{\\\"date\\\":\\\"07:41:37 GMT+0000 (Coordinated Universal Time)\\\",\\\"microtime\\\":1588232497184,\\\"message\\\":null,\\\"type\\\":\\\"devastator\\\",\\\"server_addr\\\":\\\"172.20.1.18\\\",\\\"server_id\\\":\\\"xxxxxx-01.dev\\\",\\\"subject\\\":\\\"INFO : Regenerating configuration...\\\",\\\"level\\\":6,\\\"error\\\":\\\"No config found in redis\\\"}\\x82\\xA4size\\b\\xAAcompressed\\xA4text", :expected_charset=>"UTF-8"}

Any idea why it complains like this ?

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.