Hi,
I'm a bit noob in the elastic world and English is not my primary language..
But, I need some help !!!!
How can I get only new documents from a remote elasticsearch database using the "elasticsearch input plugin" ?
By default it return all documents. I add a schedule method to run every minutes. But it return only the last minute data.. It's ok but not all what I need. If the service fail it will always return just the last minute.. So, if something brake for hours, we will not receive data the time it brake... Just the last minute everytime...
There is my logstash input config:
input {
elasticsearch {
type => "api-gateway"
schedule => "*/1 * * * *"
hosts => "ElasticServerSoucre:9200"
index => "source_index"
query => '{"query": { "range": { "creationDate": { "gte": "now-1m"}}}}'
}
}