Input via Log4J SocketAppender fails due to charachter problems

I'm using Log4J v2.x SocketAppender to send logs to logstash directly. The system was previously designed to send the logs via filebeat. Since I changed the setting, logstash now cannot read my inputs.

[2019-09-26T10:06:16,255][WARN ][logstash.codecs.jsonlines] JSON parse error, original data now in message field {:error=>#<LogStash::Json::ParserError: Unexpected character (',' (code 44)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
"; line: 1, column: 2]>, :data=>",\"source\":\"/home/directfn/app/NTP/CloudSetup/jboss-eap-7.1/standalone/log/OMS_Analyzed.log\",\"type\":\"log\"}\r"}
[2019-09-26T10:06:16,256][WARN ][logstash.codecs.jsonlines] JSON parse error, original data now in message field {:error=>#<LogStash::Json::ParserError: Illegal character ((CTRL-CHAR, code 31)): only regular white space (\r, \n, \t) is allowed between tokens
 at [Source: (String)"{"input_type":"log","message":163TXN-DFLT107CACHE_LOAD_FROM_DBOMS-TRADE127.0.0.1trueLOG_MESSAGE:CACHE LOAD FROM DBTXN_TIME:1569472482178DAO:com.dfn.oms.newgen.core.persistance.dao.SysParamDAOCACHE_KEY:SYMBOL_CLEAR_PERIOD"; line: 1, column: 32]>, :data=>"{\"input_type\":\"log\",\"message\":\u001F163\u001FTXN-DFLT\u001F107\u001FCACHE_LOAD_FROM_DB\u001FOMS-TRADE\u001F127.0.0.1\u001Ftrue\u001FLOG_MESSAGE:CACHE LOAD FROM DB\u0001TXN_TIME:1569472482178\u0001DAO:com.dfn.oms.newgen.core.persistance.dao.SysParamDAO\u0001CACHE_KEY:SYMBOL_CLEAR_PERIOD\u0001\u001F"}

I'm sending the logs in UTF-8 format.
Here is the log4j configuration.

<Socket name="SocketRef" host="192.168.0.51" port="5055">
		<PatternLayout>
			<charset>"UTF-8"</charset>
			<pattern>{"input_type":"log","message":"%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p  %c (%t) %msg","source":"/home/directfn/app/NTP/CloudSetup/jboss-eap-7.1/standalone/log/OMS_Analyzed.log","type":"log"}%n</pattern>
		</PatternLayout>
	</Socket>

The problem is with the msg since it contains charachters ▼O like this.

Can anyone give me a solution?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.