Can the Multiline filter plugin handle a log file that has multi-line events occurring concurrently and interweavingly ?
Explanation
I have a complex query about multi-line logs that are interwoven with other multi-line logs. Here is a snippet of what this looks like. Note: In the below snippet, Thread 6 and Thread 7 are running simultaneously
Each time an event occurs, it is logged from start to finish over multiple log lines. However, other events can be happening simultaneously on a different thread. Therefore, the logs get mixed together. Can the Aggregate filter plugin handle interwoven logs of this type? Any feedback will be greatly appreciated.
[29-Oct-2018 15:19:41] WARNING: [pool www] child 28 said into stdout: "{start very big json line"
[29-Oct-2018 15:19:41] WARNING: [pool www] child 28 said into stdout: "CONTINUE_MARK continue very big json line"
[29-Oct-2018 15:19:41] WARNING: [pool www] child 256 said into stdout: "{unexpected json from another worker}"
[29-Oct-2018 15:19:41] WARNING: [pool www] child 28 said into stdout: "{CONTINUE_MARK end of very big json line}"
It is php-fpm log, which shows workers events. I can output it in simple format (not in JSON), but how can I aggregate these lines?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.