Send logs using logback socket appender to logstach tcp server

[msq747]

Hi all,

I am trying to send java application logs using a socket appender to a tcp logstash server. The configuration of the server is as follows:

input {
tcp {
port => 21000
mode => "server"
}
}
output {
stdout {}
}

The message I receive on logstash is as follows:

[WARN ][logstash.codecs.line ] Received an event that has a different character encoding than you configured. {:text=>"\xAC\xED\u0000\u0005", :expected_charset=>"UTF-8"}
127.0.0.1 \xAC\xED\u0000\u0005

I read somewhere online that this message might be an SSL handshake rather than the actual message. I am using ch.qos.logback.classic.net.SocketAppender to send the log message. Is there a way I can disable SSL on the logstash TCP server so that I can get the actual message?

Thanks.

[magnusbaeck (Magnus Bäck)]

The problem is that SocketAppender sends a serialized Java object rather than plain text. You can use a log4j input to deserialize it but I recommend you don't and instead dig up an appender that sends the log data as plain text and that you use it with a JSON formatter.

[msq747]

Magnus,

Thank you for the quick response. I am currently testing both the above methods. Found logstash-logback-encoder library and added modified my logstash server conf as below:

input {

tcp {
port => 21000
mode => "server"
type => log4j
codec => plain {
charset => "ISO-8859-1"
}
}
}

output {
stdout {}
}

I do see messages in UTF-8 plain text format but only a subset of them. That is if I send 5-10 requests to logstash I observe only 2-3 messages on the server side. For some reason messages are being dropped and I am investigating.