Hi,
What we did:
Migrated all Filters/input/output from LS-2.4 to LS-6.3.
Lumberjack input receives logs from AIX and Solaris clients using LSF.
All filters/plugins work well in LS-2.4.
Having problems when we run same code in LS-6.3. Same clients, same data coming to system.
Details description below:
We recently launched LS-6.3 to receive logs from AIX/Solaris clients using logstash-forwarder.
We have lumberjack input used in LS-6.3. There are many filters/plugins used like elapsed, prune,stomp etc.
We are getting error dynamically in LS. there is no exact time/pattern of the error. It seems some data is being processed randomly and filters are not working properly for the inputs coming via lumberjack in LS-6.3.
Its very difficult to debug the same at real time, as we cannot keep the cluster on hold for long.
Errors not showing exact place, or line or the filter name which is failing.
Please guide how can we debug this issue?
What I want to know either line number, or the message which is failing? Or at least the filter which is failing?
Is there any way I can make logstash print the failed message in error logs?
Regards,
Sunil.