# How logstash takes new file and sends it to elasticsearch

**URL:** https://discuss.elastic.co/t/how-logstash-takes-new-file-and-sends-it-to-elasticsearch/175770
**Category:** Logstash
**Created:** [April 8, 2019, 6:53am UTC](https://discuss.elastic.co/t/how-logstash-takes-new-file-and-sends-it-to-elasticsearch/175770 "2019-04-08T06:53:43Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![Hari\_Krishnan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hari_krishnan/32/66696_2.png) [@Hari\_Krishnan](https://discuss.elastic.co/u/Hari_Krishnan)
#### Post date: [April 8, 2019, 6:53am UTC](https://discuss.elastic.co/t/how-logstash-takes-new-file-and-sends-it-to-elasticsearch/175770/1 "2019-04-08T06:53:44Z")

</div>

Hi ,

I have a script to create a file that logstash takes as input, and i configured logstash with path for the input file like this ./logs/\*/allfiles\_decode.json

My concern is ,  
for input file creation my script takes 10 mins to complete the write operation,  
If logstash takes input file before the script complete the write operation or not ?

---

<div class="post-metadata">

### Author: ![BennyInc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bennyinc/32/21751_2.png) [@BennyInc](https://discuss.elastic.co/u/BennyInc)
#### Post date: [April 8, 2019, 7:37am UTC](https://discuss.elastic.co/t/how-logstash-takes-new-file-and-sends-it-to-elasticsearch/175770/2 "2019-04-08T07:37:11Z")

</div>

By default, Logstash would start reading the file as soon as it is created and found by Logstash. If your file contains multiple lines, it will read them line by line anyway. If you append to an existing line, I guess you would need to try that out?

---

<div class="post-metadata">

### Author: ![Hari\_Krishnan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hari_krishnan/32/66696_2.png) [@Hari\_Krishnan](https://discuss.elastic.co/u/Hari_Krishnan)
#### Post date: [April 8, 2019, 8:52am UTC](https://discuss.elastic.co/t/how-logstash-takes-new-file-and-sends-it-to-elasticsearch/175770/3 "2019-04-08T08:52:49Z")

</div>

Thank you bennyInc,

Ok i do the experiment, i think its wrong to read the file which is in progress (I have not close the file)

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 8, 2019, 8:56am UTC](https://discuss.elastic.co/t/how-logstash-takes-new-file-and-sends-it-to-elasticsearch/175770/4 "2019-04-08T08:56:09Z")

</div>

Logstash is designed to tail files that are constantly appended to, which is the behaviour you are seeing. If you do not want it to start processing the file until it is complete, why not write it in a temporary directory and then move it once it is complete so Logstash can find it at that point?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 6, 2019, 8:56am UTC](https://discuss.elastic.co/t/how-logstash-takes-new-file-and-sends-it-to-elasticsearch/175770/5 "2019-05-06T08:56:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
