# Handle logs comming only when Logstash is turned on

**URL:** <https://discuss.elastic.co/t/handle-logs-comming-only-when-logstash-is-turned-on/62794>\
**Category:** Logstash\
**Created:** [October 12, 2016, 9:39am UTC](https://discuss.elastic.co/t/handle-logs-comming-only-when-logstash-is-turned-on/62794 "2016-10-12T09:39:29Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![krzysztof\_pl](https://avatars.discourse-cdn.com/v4/letter/k/eada6e/32.png) [@krzysztof\_pl](https://discuss.elastic.co/u/krzysztof_pl)\
**Post date:** [October 12, 2016, 9:39am UTC](https://discuss.elastic.co/t/handle-logs-comming-only-when-logstash-is-turned-on/62794/1 "2016-10-12T09:39:29Z")

</div>

Hi Everyone,  
I am new one here. I have got a question regarding to 'file input' plugin and sincedb file. Let me try to describe situation:  
Traces are coming all the time to the specific log file. Generally Logstash is turned on,  
but sometimes I would like to turn it off. In this case I would like to skip traces, I do not need to take care of them. When I turn Logstash on again I wish to process only new incoming traces.

So my question is: how to handle logs comming only when Logstash is turned on?  
Should I manipulate/update sincedb file? Is it the only proper solution?  
Maybe there is certain flag to deal with it in this way?  
Please correct me if I am wrong, but it seems that default value of start\_position ('end') does not fit in this case.

I really appreciate. Any suggestions are welcomed.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 12, 2016, 9:45am UTC](https://discuss.elastic.co/t/handle-logs-comming-only-when-logstash-is-turned-on/62794/2 "2016-10-12T09:45:47Z")

</div>

If the program writing to the log file picks up when the file has been rotated you could rotate the file immediately before Logstash starts. Another option is to throw away events older than Logstash's start time. You can probably find the start time using a ruby filter but a simpler option is probably to run Logstash with an autogenerated configuration file that contains a filter that drops too old events.

---

<div class="post-metadata">

**Author:** ![krzysztof\_pl](https://avatars.discourse-cdn.com/v4/letter/k/eada6e/32.png) [@krzysztof\_pl](https://discuss.elastic.co/u/krzysztof_pl)\
**Post date:** [October 12, 2016, 11:37am UTC](https://discuss.elastic.co/t/handle-logs-comming-only-when-logstash-is-turned-on/62794/3 "2016-10-12T11:37:12Z")

</div>

Thank you. I will try approach as you described: " to throw away events older than Logstash start (...)".

---

<div class="post-metadata">

**Author:** ![krzysztof\_pl](https://avatars.discourse-cdn.com/v4/letter/k/eada6e/32.png) [@krzysztof\_pl](https://discuss.elastic.co/u/krzysztof_pl)\
**Post date:** [October 18, 2016, 7:41am UTC](https://discuss.elastic.co/t/handle-logs-comming-only-when-logstash-is-turned-on/62794/4 "2016-10-18T07:41:43Z")

</div>

I have got another proposition:  
In 'file' input plugin I set: start\_position =\> "end".  
Each time when I start Logstash I remove ".sincedb\_\*\*\*\*" file.  
At first sight it works perfectly fine.

Maybe it would be helpful for other people.  
Btw: Are you aware of any downsides related with this solution?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 18, 2016, 7:56am UTC](https://discuss.elastic.co/t/handle-logs-comming-only-when-logstash-is-turned-on/62794/5 "2016-10-18T07:56:41Z")

</div>

Ah, right. Yes, that's a very clean solution.

In that case you should use the `sincedb_path` option to pick an exact path for the sincedb file so that you a) don't hardcode the name of the autogenerated file and b) don't have to delete $HOME/.sincedb\_\*.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:33am UTC](https://discuss.elastic.co/t/handle-logs-comming-only-when-logstash-is-turned-on/62794/6 "2017-07-06T04:33:47Z")

</div>


