Hello All,
I am facing one problem with kafka logstash plugin, I have two kafka topic (ie. Topic-1, topic-2) and two indexes on Elasticsearch (ie. Index-1, index-2). I want to send topic-1 data to index-1 and topic-2 data to index-2.
For that I have written 2 conf file, placed them in single directory and make a pipeline for that directory.
When there is any data comes to topic-1, this data is sends to both the indexes instead of index-1 only.
Below is my logstash conf file:
input {
kafka {
consumer_threads => 1
topics => "room"
bootstrap_servers =>"localhost:9092"
group_id => "room"
codec => "json"
decorate_events => true
}
}
output {
elasticsearch {
hosts => "localhost:9200"
index => "room"
document_type => "_doc"
workers => 1
}
stdout { codec => rubydebug }
}
Second Config file:
input {
kafka {
consumer_threads => 1
topics => "student"
bootstrap_servers =>"localhost:9092"
group_id => "student"
codec => "json"
decorate_events => true
}
}
output {
elasticsearch {
hosts => "localhost:9200"
index => "student"
document_type => "_doc"
workers => 1
}
stdout { codec => rubydebug }
}
I want to send each topic data to related single index only.