# Feed Logstash with gzipped multiline inputs **URL:** \ **Category:** Logstash\ **Created:** [July 13, 2017, 8:35am UTC](https://discuss.elastic.co/t/feed-logstash-with-gzipped-multiline-inputs/92963 "2017-07-13T08:35:09Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![gilbertotcc](https://avatars.discourse-cdn.com/v4/letter/g/b5e925/32.png) [@gilbertotcc](https://discuss.elastic.co/u/gilbertotcc)\ **Post date:** [July 13, 2017, 8:35am UTC](https://discuss.elastic.co/t/feed-logstash-with-gzipped-multiline-inputs/92963/1 "2017-07-13T08:35:09Z")
Hi All, I'm implementing an ELK stack for Java logs indexing and analysis. At the moment, it's only a proof of concept and I cannot feed Logstash with plain text log files; I've got gzipped past log files and I want to index their content. Given that I cannot use multiple codecs (`gzip_lines` and `multiline`), which is the best solution to index them? Should I aggregate lines and then feed Logstash with the result? Or are there other ways to reach my goal? I've written a Python script to read lines from GZIP files and feed Logstash via `http_input` plugin but I suppose it is not the best solution (according to the long times needed to index files). Thank you in advance for suggestions. ---
**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\ **Post date:** [July 21, 2017, 8:41am UTC](https://discuss.elastic.co/t/feed-logstash-with-gzipped-multiline-inputs/92963/2 "2017-07-21T08:41:53Z")
I suggest that you use your python script to unzip to another folder and have filebeat read the unzipped files and send them to the beats input. Filebeat has support for multiline. ---
**Author:** ![gilbertotcc](https://avatars.discourse-cdn.com/v4/letter/g/b5e925/32.png) [@gilbertotcc](https://discuss.elastic.co/u/gilbertotcc)\ **Post date:** [July 21, 2017, 10:40am UTC](https://discuss.elastic.co/t/feed-logstash-with-gzipped-multiline-inputs/92963/3 "2017-07-21T10:40:59Z")
Thank you very much for the suggestion. Meanwhile I've implemented a Java program to extract log entries —taking account of multiline entries too— and push them into a Redis list. Then Logstash takes such entries from Redis and index them into Logstash. It works but I've not measured performances yet. In this manner I know when all lines of the GZIP file are processes. In the solution you suggested is there a way to know when the file is processed? The goal is to remove the uncompressed file as soon as its content is loaded somewhere and ready to be indexed. ---
**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\ **Post date:** [July 21, 2017, 11:08am UTC](https://discuss.elastic.co/t/feed-logstash-with-gzipped-multiline-inputs/92963/4 "2017-07-21T11:08:24Z")
Your solution is fine. If you need more performance you can push alternate file contents to two redis instances and use two LS instances to read from each redis and output to the same ES index. ---
**Author:** ![gilbertotcc](https://avatars.discourse-cdn.com/v4/letter/g/b5e925/32.png) [@gilbertotcc](https://discuss.elastic.co/u/gilbertotcc)\ **Post date:** [July 21, 2017, 12:21pm UTC](https://discuss.elastic.co/t/feed-logstash-with-gzipped-multiline-inputs/92963/5 "2017-07-21T12:21:49Z")
I will try with both one and two Redis instances to evaluate performances. If the topic will be still open I'll post here the benchmark results. Thank you for your help. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 18, 2017, 12:22pm UTC](https://discuss.elastic.co/t/feed-logstash-with-gzipped-multiline-inputs/92963/6 "2017-08-18T12:22:13Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.