How logstash receive multiple topics from kafka

Elastic Stack Logstash
1

I'm trying to use logstash to receive data from kafka. I'm having this configurations in Kafka below, two topics and one groupID.

 "DeviceTopic": "device", //Topic for device-gruoped messages
 "TaskTopic": "task", //Topic for task-grouped messages
 "GroupId": "MGI-OUTTLER"

I want to know how to write the logstash configuration to enable this? I've been test this below but it can't work. I can't find a reference to tell how to configure it correctly.

input { 
    kafka {
    id => "test1"
	topics => "device"
	group_id => "MGI-OUTTLER"
  }
	kafka {
    id => "test2"
	topics => "task"
	group_id => "MGI-OUTTLER"
  }
 }

Thanks in advance!

2

https://www.elastic.co/guide/en/logstash/current/plugins-inputs-kafka.html#plugins-inputs-kafka-topics

Topic can be an array. When I tested this before I think I did something like this.

input {
    kafka {
      topics => ["device", "task"]
      group_id => "MGI-OUTTLER"
  }
}

To separte them in filter/output you would use an if statement.

if [kafka][topic] == "device" {

}
3

Thank you so much！
I've tried this, and it shows "Successufully subscribed to topics: device, tasks". Does that means the conversation is started correctly? Because I still can't get any message, but I guess it's more because of the software that I use to send the Kafka message. There might be some issue with its kafka message.
Anyway, thanks a lot!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.