Hi,
I have configured Logstash to receive from 2 Kafka topics using 2 diff conf files but it is only receiving from 1 topic. I can see messages coming on console consumer but logstash is unable to fetch it.
I am unable to understand what is the issue.
Below is the debug message that I receive continuously -
[2017-10-11T11:42:01,155][DEBUG][org.apache.kafka.clients.consumer.internals.AbstractCoordinator] Sending Heartbeat request for group logstash to coordinator 10.10 0.208.97:9092 (id: 2147483647 rack: null)
[2017-10-11T11:42:01,156][DEBUG][org.apache.kafka.clients.consumer.internals.AbstractCoordinator] Received successful Heartbeat response for group logstash
[2017-10-11T11:42:03,971][DEBUG][org.apache.kafka.clients.consumer.internals.ConsumerCoordinator] Sending asynchronous auto-commit of offsets {} for group logstash
[2017-10-11T11:42:04,069][DEBUG][org.apache.kafka.clients.consumer.internals.ConsumerCoordinator] Completed auto-commit of offsets {} for group logstash
[2017-10-11T11:42:04,169][DEBUG][org.apache.kafka.clients.consumer.internals.AbstractCoordinator] Sending Heartbeat request for group logstash to coordinator 10.10 0.208.97:9092 (id: 2147483647 rack: null)
[2017-10-11T11:42:04,170][DEBUG][org.apache.kafka.clients.consumer.internals.AbstractCoordinator] Received successful Heartbeat response for group logstash
[2017-10-11T11:42:05,896][DEBUG][logstash.pipeline ] Pushing flush onto pipeline {:pipeline_id=>"main", :thread=>"#<Thread:0x4a226753@/home/newELK/ logstash-6.0.0-rc1/logstash-core/lib/logstash/pipeline.rb:290 sleep>"}
[2017-10-11T11:42:07,184][DEBUG][org.apache.kafka.clients.consumer.internals.AbstractCoordinator] Sending Heartbeat request for group logstash to coordinator 10.10 0.208.97:9092 (id: 2147483647 rack: null)
[2017-10-11T11:42:07,185][DEBUG][org.apache.kafka.clients.consumer.internals.AbstractCoordinator] Received successful Heartbeat response for group logstash
[2017-10-11T11:42:08,971][DEBUG][org.apache.kafka.clients.consumer.internals.ConsumerCoordinator] Sending asynchronous auto-commit of offsets {} for group logstash
[2017-10-11T11:42:08,992][DEBUG][org.apache.kafka.clients.consumer.internals.ConsumerCoordinator] Completed auto-commit of offsets {} for group logstash
[2017-10-11T11:42:10,196][DEBUG][org.apache.kafka.clients.consumer.internals.AbstractCoordinator] Sending Heartbeat request for group logstash to coordinator 10.10 0.208.97:9092 (id: 2147483647 rack: null)
[2017-10-11T11:42:10,196][DEBUG][org.apache.kafka.clients.consumer.internals.AbstractCoordinator] Received successful Heartbeat response for group logstash
[2017-10-11T11:42:10,897][DEBUG][logstash.pipeline ] Pushing flush onto pipeline {:pipeline_id=>"main", :thread=>"#<Thread:0x4a226753@/home/newELK/ logstash-6.0.0-rc1/logstash-core/lib/logstash/pipeline.rb:290 sleep>"}