# Whether log stash parsed log file completely

**URL:** <https://discuss.elastic.co/t/whether-log-stash-parsed-log-file-completely/24055>\
**Category:** Logstash\
**Created:** [June 22, 2015, 6:16am UTC](https://discuss.elastic.co/t/whether-log-stash-parsed-log-file-completely/24055 "2015-06-22T06:16:16Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Siddhant\_Agarwal](https://avatars.discourse-cdn.com/v4/letter/s/ecb155/32.png) [@Siddhant\_Agarwal](https://discuss.elastic.co/u/Siddhant_Agarwal)\
**Post date:** [June 22, 2015, 6:16am UTC](https://discuss.elastic.co/t/whether-log-stash-parsed-log-file-completely/24055/1 "2015-06-22T06:16:16Z")

</div>

I want to run a script when log stash has finished parsing log file completely. Can anyone suggest a way to do that?

---

<div class="post-metadata">

**Author:** ![eperry](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eperry/32/551_2.png) [@eperry](https://discuss.elastic.co/u/eperry)\
**Post date:** [June 23, 2015, 4:14am UTC](https://discuss.elastic.co/t/whether-log-stash-parsed-log-file-completely/24055/2 "2015-06-23T04:14:05Z")

</div>

Do you have an example of your current configuration?

Logstash if using the "file" input is built for continuous reading of a file "Like tail" there is really no time it stops reading a file. It is not really set up as an Event handler.as your asking for.

You can run a script on the the contents of the last line read. If you have one ( or } ) but even that is not an event, It would kick off the script every time it occurred not just EOF

---

<div class="post-metadata">

**Author:** ![Siddhant\_Agarwal](https://avatars.discourse-cdn.com/v4/letter/s/ecb155/32.png) [@Siddhant\_Agarwal](https://discuss.elastic.co/u/Siddhant_Agarwal)\
**Post date:** [June 23, 2015, 5:44am UTC](https://discuss.elastic.co/t/whether-log-stash-parsed-log-file-completely/24055/3 "2015-06-23T05:44:41Z")

</div>

I was using the file input and was using exec output to script but it was executing at every event as mentioned by you. So I solved the problem by using stdin in input and directing log file. It finishes after parsing and then I ran the script using bash script.  
Thanks for your reply.

---

<div class="post-metadata">

**Author:** ![Siddhant\_Agarwal](https://avatars.discourse-cdn.com/v4/letter/s/ecb155/32.png) [@Siddhant\_Agarwal](https://discuss.elastic.co/u/Siddhant_Agarwal)\
**Post date:** [June 23, 2015, 9:24am UTC](https://discuss.elastic.co/t/whether-log-stash-parsed-log-file-completely/24055/4 "2015-06-23T09:24:42Z")

</div>

But this will not work if I have multiple log files or a directory of log files.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [June 23, 2015, 9:27am UTC](https://discuss.elastic.co/t/whether-log-stash-parsed-log-file-completely/24055/5 "2015-06-23T09:27:37Z")

</div>

There's no real good way to do this.

Why are you trying to do this?

---

<div class="post-metadata">

**Author:** ![Siddhant\_Agarwal](https://avatars.discourse-cdn.com/v4/letter/s/ecb155/32.png) [@Siddhant\_Agarwal](https://discuss.elastic.co/u/Siddhant_Agarwal)\
**Post date:** [June 23, 2015, 9:31am UTC](https://discuss.elastic.co/t/whether-log-stash-parsed-log-file-completely/24055/6 "2015-06-23T09:31:44Z")

</div>

I am using python API for elastic search to do some operations on various fields and putting results into elastic search. So as I want to automate this working I want to execute it after log stash has finished parsing log file/files.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 23, 2015, 10:05am UTC](https://discuss.elastic.co/t/whether-log-stash-parsed-log-file-completely/24055/7 "2015-06-23T10:05:00Z")

</div>

> [@Siddhant\_Agarwal](#):
>
> But this will not work if I have multiple log files or a directory of log files.

So... iterate over the files and invoke Logstash multiple times? Or concatenate the files?

```
cat *.log | logstash ...

```

---

<div class="post-metadata">

**Author:** ![Siddhant\_Agarwal](https://avatars.discourse-cdn.com/v4/letter/s/ecb155/32.png) [@Siddhant\_Agarwal](https://discuss.elastic.co/u/Siddhant_Agarwal)\
**Post date:** [June 23, 2015, 10:33am UTC](https://discuss.elastic.co/t/whether-log-stash-parsed-log-file-completely/24055/8 "2015-06-23T10:33:41Z")

</div>

Thanks for the reply @magnusbaeck.  
I will do following:  
cat path\_to\_log\_files/\*.log \> temp.log  
bin/logstash -f example.conf \< temp.log

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 23, 2015, 11:00am UTC](https://discuss.elastic.co/t/whether-log-stash-parsed-log-file-completely/24055/9 "2015-06-23T11:00:34Z")

</div>

Unless you need to support interruptions and recovering from them there's no point with the temporary file. Just pipe the output of `cat` straight to Logstash as I showed you.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:36am UTC](https://discuss.elastic.co/t/whether-log-stash-parsed-log-file-completely/24055/10 "2017-07-06T05:36:43Z")

</div>


