I use Redis to connect a Logstash Shipper and Indexer instance. All my filter rules are in the Indexer save for one that adds a field called "shipper" with a value representing the hostname of the node this shipper runs on. I set the "type" field on every input in the shipper.
Some messages now contain a message field which contains a string with all field names and field values I would expect, delimited by lots of "\" but no fields. For testing purposes I set the type to "redis" in the redis input in the indexer instance. These messages with the big message field have the "type" set to "redis" which should never happen, because type is set in the instance before redis for every message.
All of these strange messages have a tag called "_jsonparsefailure"
Strange thing is, I never use json only a "json_lines" in an output within in the indexer (the one after Redis) which forwards some messages to another logstash host on another host for further processing (namely sent them to Icinga Monitoring). This output to another logstash is used in parallel with the Elasticsearch output.
What leaves me totally puzzled is that many messages take the same path through the shipper, the redis, the indexer and even the output to the extra logstash instance for icinga. Put only some have this strange phenomenon that all the event is put into the message field and some others don't. I could not find a thing all changed events have in common which the others don't.
Could you give me a hint where I should look further?