# Logstash is not sending the data to elasticsearch

**URL:** <https://discuss.elastic.co/t/logstash-is-not-sending-the-data-to-elasticsearch/262365>\
**Category:** Logstash\
**Created:** [January 27, 2021, 12:05pm UTC](https://discuss.elastic.co/t/logstash-is-not-sending-the-data-to-elasticsearch/262365 "2021-01-27T12:05:08Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![sauravsuman689](https://avatars.discourse-cdn.com/v4/letter/s/90ced4/32.png) [@sauravsuman689](https://discuss.elastic.co/u/sauravsuman689)\
**Post date:** [January 27, 2021, 12:05pm UTC](https://discuss.elastic.co/t/logstash-is-not-sending-the-data-to-elasticsearch/262365/1 "2021-01-27T12:05:09Z")

</div>

```auto
Hi Team,

My dataflow is as below :

filebeat -> kafka -> logstash -> es -> kibana

Messages are coming to Kafka topic but I don't see logstash is processing anything and sending to es cluster.

Below is the warning messages I see in logstash logs.
 
**
[2021-01-27T11:19:08,908][WARN][org.apache.kafka.clients.consumer.internals.AbstractCoordinator] [Consumer clientId=logstash-2, groupId=prod_app] This member will leave the group because consumer poll timeout has expired. This means the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time processing messages. You can address this either by increasing max.poll.interval.ms or by reducing the maximum size of batches returned in poll() with max.poll.records.

**

I tried tuning logstash config with the below parameter but still getting warnings.

input {
  kafka {
    bootstrap_servers => "xxxxxxxxxx:9092"
    topics => ["prod_external", "prod_internal"]
    group_id => "prod_app"
    consumer_threads => 5
    codec => "json"
    heartbeat_interval_ms => "1000"
    max_poll_records => "100"
    poll_timeout_ms => "10000"
    request_timeout_ms => "120000"
    session_timeout_ms => "130000"

Please advise.

```

---

<div class="post-metadata">

**Author:** ![sauravsuman689](https://avatars.discourse-cdn.com/v4/letter/s/90ced4/32.png) [@sauravsuman689](https://discuss.elastic.co/u/sauravsuman689)\
**Post date:** [February 2, 2021, 5:52am UTC](https://discuss.elastic.co/t/logstash-is-not-sending-the-data-to-elasticsearch/262365/2 "2021-02-02T05:52:29Z")

</div>

Hi Team,

Can someone please advise on how to fix the above issue?

Regards,  
Saurav Suman

---

<div class="post-metadata">

**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\
**Post date:** [February 2, 2021, 3:37pm UTC](https://discuss.elastic.co/t/logstash-is-not-sending-the-data-to-elasticsearch/262365/3 "2021-02-02T15:37:49Z")

</div>

I can't give you the solution extracly but i think you can try to debug.

`filebeat -> kafka -> logstash -> es -> kibana`

1. Check new event in Kafka. If kafka still got new event from filebeat, jump to (4), or not, jump to to (2)
2. Kafka didn't new event, check filebeat, (run filebeat with debug to make sure filebeat is working as well `filebeat -e -d "*"` ). If FB worked, jump to check kafka (3)
3. Check Kafka. Did Kafka still worked ?
4. FB -\> Kafka is worked, check on logstash, run logstahs with debug or verbose mode.

I think you can find problem. Good luck!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 2, 2021, 3:38pm UTC](https://discuss.elastic.co/t/logstash-is-not-sending-the-data-to-elasticsearch/262365/4 "2021-03-02T15:38:12Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.