Hi,
We use a cloud service that I've configured to forward it's logs to a logstash instance using the tcp input.
The Logstash instance is configured to take this message and just output to Redis, which in turn will be processed by my beefier LS servers where the text is grokked etc
For the most part the logs come in and ship onto Redis fine (around 500 a second) but, one in every few thousand will trigger the following error in the eventlog:
:message=>"Received an event that has a different character encoding than you configured."
Looking at the text of the error it shows the following (truncated and ip's changed):
Sep 07 07:59:50 zscaler-nss: LEEF:1.0|Zscaler|NSS|4.1|Allowed|cat=Allowed\\tdevTime=Sep 07 2016 07:59:50 GMT\\tdevTimeFormat=MMM dd yyyy HH:mm:ss z\\tsrc=192.168.1.100\\tdst=111.111.111.111\\tsrcPostNAT=111.111.111.111\\trealm=........", :expected_charset=>"UTF-8", :level=>:warn
If I compare the output of the above with one that successfully ships, the only real difference I can see is that the above message is using '\\t' for the tab field where as the successful ones have an actual whitepaced tab.
Is there anything I can do to accept both messages? I've looked at the plain codec, but I'm worried that changing the encoding to another format will just cause more issues?
Regards
Xet