# Logstash not able to fetch messages from Amazon MSK

**URL:** https://discuss.elastic.co/t/logstash-not-able-to-fetch-messages-from-amazon-msk/250538
**Category:** Logstash
**Created:** [September 30, 2020, 4:20pm UTC](https://discuss.elastic.co/t/logstash-not-able-to-fetch-messages-from-amazon-msk/250538 "2020-09-30T16:20:36Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![Akhil96](https://avatars.discourse-cdn.com/v4/letter/a/f6c823/32.png) [@Akhil96](https://discuss.elastic.co/u/Akhil96)
#### Post date: [September 30, 2020, 4:20pm UTC](https://discuss.elastic.co/t/logstash-not-able-to-fetch-messages-from-amazon-msk/250538/1 "2020-09-30T16:20:36Z")

</div>

I am trying to consume from an MSK cluster using logstash and push the messages to an Elasticsearch instance. Due to some reason the data is not getting fetched correctly, and getting below message in logs.

Skipping fetch for partition {TOPIC}-0 because previous request to (id: 2 rack: use1-az2) has not been processed.

Below is my logstash config.  
input {  
kafka {  
bootstrap\_servers =\> "{MSK\_HOST}" topics =\> "{TOPIC}"  
group\_id =\> "${KAFKA\_GRP\_ID}"  
auto\_offset\_reset =\> "latest"  
#ssl\_truststore\_location =\> "/usr/share/logstash/config/kafka.client.truststore.jks"  
security\_protocol =\> "SSL"  
decorate\_events =\> true  
codec =\> json  
}  
}

filter {  
ruby {  
code =\> "event.set('[@metadata][kafka][lc\_topic]', event.get('[@metadata][kafka][topic]').split(/(?=[A-Z])/).map{|x| x.downcase }.join('') )"  
}  
}

output {  
amazon\_es {  
hosts =\> ["${ES\_HOST}"]  
ssl =\> true  
#ilm\_enabled =\> false  
index =\> "logstash.%{[@metadata][kafka][lc\_topic]}.%{+YYYY.MM.dd}"  
manage\_template =\> false  
aws\_access\_key\_id =\> ''  
aws\_secret\_access\_key =\> ''

}  
stdout {  
codec =\> rubydebug  
}

}

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 28, 2020, 4:20pm UTC](https://discuss.elastic.co/t/logstash-not-able-to-fetch-messages-from-amazon-msk/250538/2 "2020-10-28T16:20:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
