Kafka to Elastic search through Logstash for add, delete, update actions


(Vipin Dwivedi) #1

Hi,
I am looking a solution for loading data in form of messages with Add, Update, Delete actions from Kafka stream to Elastic search, some of the options I have explored and found simple way is through Logstash but have some questions if eom experience member can answer..

  1. How can we do add, update, delete action based on message parameter value.
  2. Performance will be based on kafka stream throughput, or should be do some buffering to manage or some Batching option is available.
    Any reference with some logstash config will be very helpful.
    thanks
    vipin

(Vipin Dwivedi) #2

Any help on this topic please, this will help us to decide if can use Logstash or need to explore some other options ..


(Magnus B├Ąck) #3

How can we do add, update, delete action based on message parameter value.

See https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html.

Performance will be based on kafka stream throughput, or should be do some buffering to manage or some Batching option is available.

Not sure exactly what you mean. Logstash is going to be slower than Kafka.


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.