# Kafka to Logstash

**URL:** <https://discuss.elastic.co/t/kafka-to-logstash/337173>\
**Category:** Logstash\
**Created:** [June 29, 2023, 10:24am UTC](https://discuss.elastic.co/t/kafka-to-logstash/337173 "2023-06-29T10:24:48Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Shalinicts](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shalinicts/32/81251_2.png) [@Shalinicts](https://discuss.elastic.co/u/Shalinicts)\
**Post date:** [June 29, 2023, 10:24am UTC](https://discuss.elastic.co/t/kafka-to-logstash/337173/1 "2023-06-29T10:24:48Z")

</div>

Hi Team ,

In Logstash I can see below error for pipeline having input as Kafka (Oracle Cloud)  
It fails to connect and then discover .  
Any idea on what needs to be checked. Once it is stable automatically, we gets data in Elastic.

**Log Sample :**  
[INFO][org.apache.kafka.clients.consumer.internals.AbstractCoordinator][oic\_xxx][0584xxxxxxx] [Consumer clientId=logstash-0, groupId=logstash] Group coordinator [xxx.oci.oraclecloud.com:9092](http://xxx.oci.oraclecloud.com:9092) (id: 2xxxx rack: null) is unavailable or invalid, will attempt rediscovery

[INFO][org.apache.kafka.clients.consumer.internals.AbstractCoordinator][oic\_xxx][058xxx] [Consumer clientId=logstash-0, groupId=logstash] Discovered group coordinator [xxx.oci.oraclecloud.com:9092](http://xxx.oci.oraclecloud.com:9092) (id: 2xxxx rack: null)

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [June 29, 2023, 11:32am UTC](https://discuss.elastic.co/t/kafka-to-logstash/337173/2 "2023-06-29T11:32:20Z")

</div>

Those are not errors, they are informational messages.

It means the Logstash temporarily lost the connection with that Kafka broker and reconnected.

This is more a network/infrastructure issue.

What is the time difference between those two messages? You didn't share the full log line, so it is not possible to know.

---

<div class="post-metadata">

**Author:** ![Shalinicts](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shalinicts/32/81251_2.png) [@Shalinicts](https://discuss.elastic.co/u/Shalinicts)\
**Post date:** [June 29, 2023, 11:47am UTC](https://discuss.elastic.co/t/kafka-to-logstash/337173/3 "2023-06-29T11:47:42Z")

</div>

Thanks Leandro Pereira.  
Any idea on what can be done or what needs to be checked?  
The time difference between messages are within milliseconds

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [June 29, 2023, 12:24pm UTC](https://discuss.elastic.co/t/kafka-to-logstash/337173/4 "2023-06-29T12:24:25Z")

</div>

> [@Shalinicts](#):
>
> Any idea on what can be done or what needs to be checked?  
> The time difference between messages are within milliseconds

Not sure, are you connecting to your Kafka using the public internet? From the log you shared it seems to be exposed to the internet: `xxx.oci.oraclecloud.com`.

Many cans cause network issues.

I do not see this as a problem since the connection it is reestablished after some miliseconds.

Is this happening frequently at the point of impact your ingestion?

---

<div class="post-metadata">

**Author:** ![Shalinicts](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shalinicts/32/81251_2.png) [@Shalinicts](https://discuss.elastic.co/u/Shalinicts)\
**Post date:** [June 29, 2023, 12:58pm UTC](https://discuss.elastic.co/t/kafka-to-logstash/337173/5 "2023-06-29T12:58:33Z")

</div>

We see this recently for every one hour and no data is ingested.

**id: 2xxxx rack: null** -- Does this means no data in topic ?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 27, 2023, 12:58pm UTC](https://discuss.elastic.co/t/kafka-to-logstash/337173/6 "2023-07-27T12:58:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
