Dear Logstash/ElasticSearch,
With logstash I'm reading an event file.
Each event has a timestamp with it, and several events are relating to one document in elastic search.
Typically events are
1° "started" ,
2° "doing somethign" ,
3° "failed" of "delivered"
The issue is that the events will not necessary come in the correct order and I need to ensure that the event I send to Elastic seach is newer then the last one I sent.
I added a field 'LastEventTimestamp" in ElasticSearch for each document so should be easy enough.( or so I thought)
However I can't seem to get the combination in one logstach configuration.
1° reading from event file
2° lookup in elastic search for Document and taking the latest LastEventTimestamp
3° updating Document if date is ok
The lookup part is eluding me.
Any help welcome !
Kind regards
Laurens