# Scheduling a logsatsh config file **URL:** \ **Category:** Logstash\ **Created:** [December 2, 2016, 6:31am UTC](https://discuss.elastic.co/t/scheduling-a-logsatsh-config-file/67831 "2016-12-02T06:31:50Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![sabyasachi\_mallick](https://avatars.discourse-cdn.com/v4/letter/s/90db22/32.png) [@sabyasachi\_mallick](https://discuss.elastic.co/u/sabyasachi_mallick)\ **Post date:** [December 2, 2016, 6:31am UTC](https://discuss.elastic.co/t/scheduling-a-logsatsh-config-file/67831/1 "2016-12-02T06:31:50Z")
i have 4 independent logstash config files which are taking input from a csv file .i am executing 4 files in background by using logstash -f /path/to/config file. But when my csv file is updated with new data,then all the configurations accessing at same time , for which at the same time all are trying to load data into elastic . For which my elastic server is going down due to unable to handle these many requestes at same time. i know for jdbc, scheduling is available but for csv plugin , i did not get any info. Is there any way to schedule logstash config files to run in a particualr time or any other solution to overcome this problem? any help ll be appreciated!! thanks ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 2, 2016, 6:57am UTC](https://discuss.elastic.co/t/scheduling-a-logsatsh-config-file/67831/2 "2016-12-02T06:57:46Z")
> For which my elastic server is going down due to unable to handle these many requestes at same time. It sounds like you have an underpowered ES server. Does it run out of heap, or why does it crash? > Is there any way to schedule logstash config files to run in a particualr time or any other solution to overcome this problem? You could use cron, but I think you're trying to solve the wrong problem. Logstash and its file input is meant to continuously monitor files, not batch process files periodically. ---
**Author:** ![sabyasachi\_mallick](https://avatars.discourse-cdn.com/v4/letter/s/90db22/32.png) [@sabyasachi\_mallick](https://discuss.elastic.co/u/sabyasachi_mallick)\ **Post date:** [December 2, 2016, 7:04am UTC](https://discuss.elastic.co/t/scheduling-a-logsatsh-config-file/67831/3 "2016-12-02T07:04:02Z")
yes. i allocated 1 GB heap to elastic . since all the config files sending requests to my elastic server at same time, for that it is going down with out of heap memory . I want to send the config file's requests one after another , after updating csv file. if i ll do cron job to run logstash config files, then since\_db in csv plugin will create a problem, because each time i ll run a logsatsh csv plugin config file, it ll try to create a new since\_db , since it is already present,so it ll not execute. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 2, 2016, 7:09am UTC](https://discuss.elastic.co/t/scheduling-a-logsatsh-config-file/67831/4 "2016-12-02T07:09:32Z")
> yes. i allocated 1 GB heap to elastic . since all the config files sending requests to my elastic server at same time, for that it is going down with out of heap memory . I want to send the config file's requests one after another , after updating csv file. Your ES server is underpowered. I strongly suggest you solve that problem instead of working around it by serializing requests. > if i ll do cron job to run logstash config files, then since\_db in csv plugin will create a problem, because each time i ll run a logsatsh csv plugin config file, it ll try to create a new since\_db , since it is already present,so it ll not execute. If you want multiple Logstash instances to process the same file they need to have different sincedb files. Use the file input's `sincedb_path` to explicitly set the per-instance path. ---
**Author:** ![sabyasachi\_mallick](https://avatars.discourse-cdn.com/v4/letter/s/90db22/32.png) [@sabyasachi\_mallick](https://discuss.elastic.co/u/sabyasachi_mallick)\ **Post date:** [December 2, 2016, 9:24am UTC](https://discuss.elastic.co/t/scheduling-a-logsatsh-config-file/67831/5 "2016-12-02T09:24:52Z")
Thanks @magnusbaeck for your quick response. Yes ! my elasticsearch is underpowered. only 1 gb heap size i allocated. i'll increase the heap size . for time being , i solved this issue by writing 4 of elastic search index in same logstash file in series. so tht it ll send the file ll send request one after another. And can you please explain little bit the last line "If you want multiple Logstash instances to process the same file they need to have different sincedb files. Use the file input's sincedb\_path to explicitly set the per-instance path." it seems interesting. i never did this..can you please explain this? Thanks ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 2, 2016, 9:45am UTC](https://discuss.elastic.co/t/scheduling-a-logsatsh-config-file/67831/6 "2016-12-02T09:45:10Z")
> And can you please explain little bit the last line "If you want multiple Logstash instances to process the same file they need to have different sincedb files. Use the file input's sincedb\_path to explicitly set the per-instance path." > it seems interesting. i never did this..can you please explain this? I'm not sure what's unclear. The `sincedb_path` option should be pretty straight-forward. ---
**Author:** ![sabyasachi\_mallick](https://avatars.discourse-cdn.com/v4/letter/s/90db22/32.png) [@sabyasachi\_mallick](https://discuss.elastic.co/u/sabyasachi_mallick)\ **Post date:** [December 2, 2016, 10:38am UTC](https://discuss.elastic.co/t/scheduling-a-logsatsh-config-file/67831/7 "2016-12-02T10:38:13Z")
yeaa , for each logstash config file i am using differnet since\_db path. But if i ll use cron job then, the problem will arise i.e let cron job has executed config file today , for which since\_db is created. when it ll run the same config file tomorrow , the old since\_db file ll prevent the config file to run, because it ll try to run the same config file with existing since\_db. Thats what i am asking. Can we avoid that since\_db path? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 2, 2016, 11:05am UTC](https://discuss.elastic.co/t/scheduling-a-logsatsh-config-file/67831/8 "2016-12-02T11:05:42Z")
As I said, if you want multiple Logstash instances to process the same file they need to have different sincedb files. Use the file input's `sincedb_path` to explicitly set the per-instance path. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 30, 2016, 11:05am UTC](https://discuss.elastic.co/t/scheduling-a-logsatsh-config-file/67831/9 "2016-12-30T11:05:42Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.