# Logstash File input Observation **URL:** \ **Category:** Logstash\ **Created:** [December 14, 2015, 3:40pm UTC](https://discuss.elastic.co/t/logstash-file-input-observation/37139 "2015-12-14T15:40:24Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![PKD](https://avatars.discourse-cdn.com/v4/letter/p/8c91f0/32.png) [@PKD](https://discuss.elastic.co/u/PKD)\ **Post date:** [December 14, 2015, 3:40pm UTC](https://discuss.elastic.co/t/logstash-file-input-observation/37139/1 "2015-12-14T15:40:24Z")
Hi all, We have configured logstash with file input to scan files from specified directory and index into elastic search. We have provided the following path for file input : /var/log/varnish/varnish-access\*.log Varnish generates log files for every hour. We have changed the stat\_interval for file input for 1 hr. stat\_interval =\> 3600 rest all parameters are by default. Our observation is that when we set stat\_interval to 1Hr the files there is a lag between the ingestion. i.e. files earlier than 6 Hrs are getting scanned. We checked with the lsof command so only the files earlier than 5-6 Hrs are opened by logstash. even if we set the stat\_interval to 1Hr. Is there any delay in the scanning (discovering the new files) and processing if we set stat\_interval to higher value ? Also we tried to change stat\_interval to 1 min then the logstash was able to pickup the files faster. Is there any relation between the stat\_interval, since\_db\_write interval ? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 14, 2015, 6:42pm UTC](https://discuss.elastic.co/t/logstash-file-input-observation/37139/2 "2015-12-14T18:42:09Z")
> Also we tried to change stat\_interval to 1 min then the logstash was able to pickup the files faster. Yes, of course. The whole purpose of the `stat_interval` option is to select how often Logstash checks if the files have been updated. If you can explain why you're exploring absurdly long `stat_interval` values maybe we can provide better answers. > Is there any relation between the stat\_interval, since\_db\_write interval ? Well, there is a relation in the sense that as long as Logstash can keep up with the data being written to the monitored files the sincedb file will never be updated more frequently than `stat_interval`. ---
**Author:** ![PKD](https://avatars.discourse-cdn.com/v4/letter/p/8c91f0/32.png) [@PKD](https://discuss.elastic.co/u/PKD)\ **Post date:** [December 15, 2015, 5:46am UTC](https://discuss.elastic.co/t/logstash-file-input-observation/37139/3 "2015-12-15T05:46:14Z")
The stat\_interval will check only the updated files. But has it any relation with discovering new files late even if we keep the longer stat\_interval ? Even after 5-6 hours it is able to discover new files, even we are using the default value for discover\_interval i.e. 15 (option to discover new files to watch) As new log files are generated every hour so we have kept it to 1 Hr. Also to reduce the stat for every file. ---
**Author:** ![wiibaa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wiibaa/32/44931_2.png) [@wiibaa](https://discuss.elastic.co/u/wiibaa)\ **Post date:** [December 15, 2015, 5:55am UTC](https://discuss.elastic.co/t/logstash-file-input-observation/37139/4 "2015-12-15T05:55:00Z")
Sorry to respond with source code to your question, but to understand precisely the relationship between the two configuration, you need to read **the main loop** that drives the input file > And the response is yes! there is a relationship between the two the concrete discover interval is stat\_interval \* discover\_interval. For sure there is room for improvment, but clarification in the documentation should come first. Thanks for reminding me about this old problem 😃 ---
**Author:** ![PKD](https://avatars.discourse-cdn.com/v4/letter/p/8c91f0/32.png) [@PKD](https://discuss.elastic.co/u/PKD)\ **Post date:** [December 15, 2015, 9:51am UTC](https://discuss.elastic.co/t/logstash-file-input-observation/37139/5 "2015-12-15T09:51:35Z")
Thanks a lot for the reply. This was very helpful. So this mean that If we have stat\_interval =\> 3600 discover\_interval =\> 15 (default for file input) So logstash will not discover any files for 15 Hrs. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 5:18am UTC](https://discuss.elastic.co/t/logstash-file-input-observation/37139/6 "2017-07-06T05:18:24Z")