Multiple kafka clusters?

Elastic Stack Logstash
1

Is it possible to have a single logstash instance listen to multiple kafka clusters?

I'm uncertain if:

input {
kafka {
bootstrap_servers => "kafka_cluster_A-01:9092,kafka_cluster_A-02:9092"
}
}

input {
kafka {
bootstrap_servers => "kafka_cluster_B-01:9092,kafka_cluster_B-02:9092"
}
}

or

input {
kafka {
bootstrap_servers => "kafka_cluster_A-01:9092,kafka_cluster_A-02:9092"
}
kafka {
bootstrap_servers => "kafka_cluster_B-01:9092,kafka_cluster_B-02:9092"
}
}

reads from multiple clusters. Do either of these configurations do what I want? Or do I need to start a separate logstash instance for each kafka cluster? (I suppose another possibility would be to aggregate multiple kafka topics onto a single central Kafka cluster which logstash reads from, but that's outside logstash configuration)

2

Okay, I figured out that what I want to do is best implemented through logstash pipelines.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.