Schedule job for elasticsearch

[prabhu_pandurangan]

Hi Team

i would like to setup schedule job for elasticsearch input plugin? how to do it?

[magnusbaeck (Magnus Bäck)]

There's nothing built-in for this (see https://github.com/logstash-plugins/logstash-input-elasticsearch/issues/32) but you can always fire up Logstash via crontab.

[franck.lefebure]

I have a customized plugin to do that
I can do a fork in github if you want to build on it

[franck.lefebure]

So it won't be lost.. : GitHub - flefebure/logstash-input-elasticsearch
It's inspired from logstash input jdbc plugin

configuration example :

elasticsearch {
hosts => "forge.local"
record_last_run => true
tracking_field => "computation"
last_run_metadata_path => "/etc/logstash/states/my.state"
query => '{"query":{"query_string":{"query":
"+_type:activity +computation:>=&elastic_last_value +computation:<now-1h"}},
"sort": ["computation" ]}'
size => 1000
index => "index_201710,index_201711"
schedule => "* * * * *"
legacy_scrolling => true
}

legacy_scrolling is to scroll data from ES1.x server