I'm implementing an ELK stack for Java logs indexing and analysis. At the moment, it's only a proof of concept and I cannot feed Logstash with plain text log files; I've got gzipped past log files and I want to index their content.
Given that I cannot use multiple codecs (
multiline), which is the best solution to index them? Should I aggregate lines and then feed Logstash with the result? Or are there other ways to reach my goal?
I've written a Python script to read lines from GZIP files and feed Logstash via
http_input plugin but I suppose it is not the best solution (according to the long times needed to index files).
Thank you in advance for suggestions.