# Logstash gzip\_lines , tail latest written file to text file

**URL:** <https://discuss.elastic.co/t/logstash-gzip-lines-tail-latest-written-file-to-text-file/99926>\
**Category:** Logstash\
**Created:** [September 9, 2017, 9:41am UTC](https://discuss.elastic.co/t/logstash-gzip-lines-tail-latest-written-file-to-text-file/99926 "2017-09-09T09:41:58Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![shashwat](https://avatars.discourse-cdn.com/v4/letter/s/bbe5ce/32.png) [@shashwat](https://discuss.elastic.co/u/shashwat)\
**Post date:** [September 9, 2017, 9:41am UTC](https://discuss.elastic.co/t/logstash-gzip-lines-tail-latest-written-file-to-text-file/99926/1 "2017-09-09T09:41:58Z")

</div>

Hi,

I am using logstash 5.5. I am using gzip\_lines plugin to parse the archived files. As , this plugin expects a text file having archived file with absolute path line by line. If this text file is static , i.e. if I write path for all the archived files in it and run logstash it works fine.

BUT , when I write this text file at the run time every minute with the new path of archived file, logstash does not process the latest file being written in the .txt file. I tried with start\_position="end" but that did not help. If I use start\_position="beginning", it re-process all the files written in the txt file and that I don't want.

I just want to process latest file being written in the txt file. Is there any way to achieve this?

Thanks,

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [September 12, 2017, 3:28am UTC](https://discuss.elastic.co/t/logstash-gzip-lines-tail-latest-written-file-to-text-file/99926/2 "2017-09-12T03:28:32Z")

</div>

You mean you're updating the gzipped log file while Logstash is running? Why? I wouldn't expect that to work.

---

<div class="post-metadata">

**Author:** ![shashwat](https://avatars.discourse-cdn.com/v4/letter/s/bbe5ce/32.png) [@shashwat](https://discuss.elastic.co/u/shashwat)\
**Post date:** [September 12, 2017, 6:15am UTC](https://discuss.elastic.co/t/logstash-gzip-lines-tail-latest-written-file-to-text-file/99926/3 "2017-09-12T06:15:33Z")

</div>

Hi Magnus,

I have these .csv.gz files coming every minute. I need to analyze them using Kibana. As per my understanding and the information I found in different forums, if we want to use gzip\_lines plugin, we need to have a .txt file with the path of .gz files in it . That's why , I was trying to update the .txt file every minute with the path of the received .gz file.  
However; another way of solving my problem can be by extracting the received .gz files in some location and then parse it using logstash/Kibana.

Thanks,

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 10, 2017, 6:15am UTC](https://discuss.elastic.co/t/logstash-gzip-lines-tail-latest-written-file-to-text-file/99926/4 "2017-10-10T06:15:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.