Need your guidance for my problem
we push the updates into our ES Index from our source, and we want to pick up those updates and further send to Kafka topic using kafka output plugin
Question: How to pick up only the updates and send, is there a way to identify the updates(new record update, update to existing record update, delete event update)
Please help guide
The Solution i am looking for is:-
To only pick up UPDATES not the whole of index data everytime we schedule and send to kafka topic - below data sends all of data every time we send
Kafka output plugin code
hosts => "xxxxxxxxxxxx01:9201"
index => "testcaseindex"
size => 1000
scroll => "5m"
docinfo => true
codec => json
topic_id => "POC.CASE.TOPIC"
bootstrap_servers => "XXXXXXXX01.XX.XXXX.com:9092"
RUN with below
bin/logstash -f logstash-filter.conf