# Logstash daily indexing

**URL:** <https://discuss.elastic.co/t/logstash-daily-indexing/124259>\
**Category:** Logstash\
**Created:** [March 16, 2018, 10:07am UTC](https://discuss.elastic.co/t/logstash-daily-indexing/124259 "2018-03-16T10:07:10Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![sana](https://avatars.discourse-cdn.com/v4/letter/s/ba9def/32.png) [@sana](https://discuss.elastic.co/u/sana)\
**Post date:** [March 16, 2018, 10:07am UTC](https://discuss.elastic.co/t/logstash-daily-indexing/124259/1 "2018-03-16T10:07:10Z")

</div>

i have a few folders containes log files and each end of the day a new file is added to each folder  
my configuration file is this:  
file {

```
path => "/home/sana/pfe/docker/logs/**/*.log"
type => "syslog"
start_position => "beginning"
ignore_older => 0

```

codec =\> multiline {  
pattern =\> "^(%{TIMESTAMP\_ISO8601})"  
negate =\> true  
what =\> "previous"  
}  
}

}

the old files are indexed to elastic search but new files are not  
even if i put this command in terminal  
sudo /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf --config.reload.automatic  
does logstash pipeline work automaticly to detect new data or i have to restart it each time

---

<div class="post-metadata">

**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\
**Post date:** [March 17, 2018, 5:44am UTC](https://discuss.elastic.co/t/logstash-daily-indexing/124259/2 "2018-03-17T05:44:56Z")

</div>

> [@sana](#):
>
> sudo /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf

Eep -- you shouldn't have to be running Logstash as the super user; doing so gives it far more control of your system than it should have, especially as it can run arbitrary plugins.

Logstash's [file input](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html) should pick up new files in the given directory as they show up.

---

<div class="post-metadata">

**Author:** ![sana](https://avatars.discourse-cdn.com/v4/letter/s/ba9def/32.png) [@sana](https://discuss.elastic.co/u/sana)\
**Post date:** [March 21, 2018, 11:31am UTC](https://discuss.elastic.co/t/logstash-daily-indexing/124259/3 "2018-03-21T11:31:08Z")

</div>

thank you for your reply, i will avoid using sudo from now on, i would like to ask if i want logstash to bring only new log event to elasticsearch because the old files are already parsed, do u put those 3 lines or not:  
start\_position =\> "beginning"  
sincedb\_path =\> "/dev/null"  
ignore\_older =\> 0

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 18, 2018, 11:31am UTC](https://discuss.elastic.co/t/logstash-daily-indexing/124259/4 "2018-04-18T11:31:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.