# Logstash Scheduler

**URL:** <https://discuss.elastic.co/t/logstash-scheduler/296431>\
**Category:** Logstash\
**Created:** [February 7, 2022, 7:53am UTC](https://discuss.elastic.co/t/logstash-scheduler/296431 "2022-02-07T07:53:27Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![shharukh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shharukh/32/139321_2.png) [@shharukh](https://discuss.elastic.co/u/shharukh)\
**Post date:** [February 7, 2022, 7:53am UTC](https://discuss.elastic.co/t/logstash-scheduler/296431/1 "2022-02-07T07:53:27Z")

</div>

Hey Elasticworld,

I'm current Logstash version 7.9 ,I have a encounter a problem in Logstash, in which have a 4 pipeline which are using "JBDC" driver for inserting data into Elasticsearch. \*\*Now I have to add another pipeline through which I could insert the data in Elasticsearch by a "CSV" on a regular basis. \*\*  
I have configured the config file it Run perfectly , but I have to Schedule it at once a day at 1 am every day along with other Scheduler which are set on a different time (for example 3 am and 4am) Can anyone help me out, How can I add a Scheduler in the following added config.

I have seen your post regarding this issue.

```auto
[type or paste code here](https://discuss.elastic.co/t/scheduling-a-logsatsh-config-file/67831/4)

```

```auto
input {
  file {
    type => "CSV"
    path => "C:\Users\Administrator\Desktop\abc\**.csv"
    start_position => "beginning"
   sincedb_path => "/dev/null"
    close_older =>"10 second"
    exit_after_read => true
    mode => "read"

  }
}
filter {
  csv {
      separator => ","
  }
}

output {
    elasticsearch {
      hosts => ["http:XXXXXXX:9200"]
    index => ["ABC"]
      user => "XXXXX"
      password =>"XXXXX"
  }
stdout {}
}

```

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [February 7, 2022, 8:19am UTC](https://discuss.elastic.co/t/logstash-scheduler/296431/2 "2022-02-07T08:19:26Z")

</div>

The file input doesn't have a schedule, it'll always watch the pattern you define and watch for new files.

Will that not work for you?

---

<div class="post-metadata">

**Author:** ![shharukh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shharukh/32/139321_2.png) [@shharukh](https://discuss.elastic.co/u/shharukh)\
**Post date:** [February 7, 2022, 10:27am UTC](https://discuss.elastic.co/t/logstash-scheduler/296431/3 "2022-02-07T10:27:39Z")

</div>

Hey @warkolm, thank you for your response.

Yes it always watch for a new file but if this process remain open then other corn job would not run as a logstash file is still in use.

I hope my explanation is understandable.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [February 7, 2022, 9:16pm UTC](https://discuss.elastic.co/t/logstash-scheduler/296431/4 "2022-02-07T21:16:10Z")

</div>

If you're already using cron, then just use it for this.  
Change the file input to stdin and then cat the file in.

---

<div class="post-metadata">

**Author:** ![shharukh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shharukh/32/139321_2.png) [@shharukh](https://discuss.elastic.co/u/shharukh)\
**Post date:** [February 8, 2022, 6:12am UTC](https://discuss.elastic.co/t/logstash-scheduler/296431/5 "2022-02-08T06:12:28Z")

</div>

@warkolm by cron I mean other Scheduler place in different config  
which are being called by pipeline.yml ( etc/logstash)  
Now I have to add a new config to pipeline.yml in which I can read a CSV from a file location mention above on a daily basis and save it to Elasticsearch.

Secondly  
Can we apply CSV filter in stdin?

I have tired it out while the logstash is running as service but it fails and give a error that another logstash instance is in used.

```auto
$ cat file | ./bin/logstash -f /bin/logstashmyconfig.conf

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 8, 2022, 6:13am UTC](https://discuss.elastic.co/t/logstash-scheduler/296431/6 "2022-03-08T06:13:10Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.