# Filebeat Stops Processing logs for AWS module

**URL:** https://discuss.elastic.co/t/filebeat-stops-processing-logs-for-aws-module/275993
**Category:** Beats
**Tags:** filebeat
**Created:** [June 15, 2021, 12:05pm UTC](https://discuss.elastic.co/t/filebeat-stops-processing-logs-for-aws-module/275993 "2021-06-15T12:05:03Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![ankitdevnalkar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ankitdevnalkar/32/46158_2.png) [@ankitdevnalkar](https://discuss.elastic.co/u/ankitdevnalkar)
#### Post date: [June 15, 2021, 12:05pm UTC](https://discuss.elastic.co/t/filebeat-stops-processing-logs-for-aws-module/275993/1 "2021-06-15T12:05:03Z")

</div>

I am facing exactly the same issue as this thread [Filebeat stops proccessing s3 input with no error](https://discuss.elastic.co/t/filebeat-stops-proccessing-s3-input-with-no-error/212590).

Also, I experienced sometimes Filebeat suddenly stops processing data for other modules too, I faced it with Gsuite module as well. However, it shows no error while `log.level` set to `debug`

here are my logs :

```auto
Jun 15 11:56:46 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:46.938Z WARN [input.s3] s3/collector.go:189 Half of the set visibilityTimeout passed, visibility timeout needs to be updated {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:46 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:46.938Z WARN [input.s3] s3/collector.go:189 Half of the set visibilityTimeout passed, visibility timeout needs to be updated {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:46 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:46.938Z WARN [input.s3] s3/collector.go:189 Half of the set visibilityTimeout passed, visibility timeout needs to be updated {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:46 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:46.939Z WARN [input.s3] s3/collector.go:189 Half of the set visibilityTimeout passed, visibility timeout needs to be updated {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:46 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:46.939Z WARN [input.s3] s3/collector.go:189 Half of the set visibilityTimeout passed, visibility timeout needs to be updated {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:46 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:46.939Z WARN [input.s3] s3/collector.go:189 Half of the set visibilityTimeout passed, visibility timeout needs to be updated {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:46 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:46.939Z WARN [input.s3] s3/collector.go:189 Half of the set visibilityTimeout passed, visibility timeout needs to be updated {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:46 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:46.940Z WARN [input.s3] s3/collector.go:189 Half of the set visibilityTimeout passed, visibility timeout needs to be updated {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:46 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:46.941Z WARN [input.s3] s3/collector.go:189 Half of the set visibilityTimeout passed, visibility timeout needs to be updated {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:46 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:46.942Z WARN [input.s3] s3/collector.go:189 Half of the set visibilityTimeout passed, visibility timeout needs to be updated {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:46 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:46.946Z WARN [input.s3] s3/collector.go:189 Half of the set visibilityTimeout passed, visibility timeout needs to be updated {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:46 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:46.949Z WARN [input.s3] s3/collector.go:189 Half of the set visibilityTimeout passed, visibility timeout needs to be updated {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:46 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:46.949Z WARN [input.s3] s3/collector.go:189 Half of the set visibilityTimeout passed, visibility timeout needs to be updated {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:46 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:46.950Z WARN [input.s3] s3/collector.go:189 Half of the set visibilityTimeout passed, visibility timeout needs to be updated {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:46 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:46.950Z WARN [input.s3] s3/collector.go:189 Half of the set visibilityTimeout passed, visibility timeout needs to be updated {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:46 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:46.950Z WARN [input.s3] s3/collector.go:189 Half of the set visibilityTimeout passed, visibility timeout needs to be updated {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:46 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:46.950Z WARN [input.s3] s3/collector.go:189 Half of the set visibilityTimeout passed, visibility timeout needs to be updated {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:46 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:46.950Z WARN [input.s3] s3/collector.go:189 Half of the set visibilityTimeout passed, visibility timeout needs to be updated {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:46 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:46.950Z WARN [input.s3] s3/collector.go:189 Half of the set visibilityTimeout passed, visibility timeout needs to be updated {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:46 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:46.950Z WARN [input.s3] s3/collector.go:189 Half of the set visibilityTimeout passed, visibility timeout needs to be updated {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:46 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:46.976Z INFO [input.s3] s3/collector.go:196 Message visibility timeout updated to 300 seconds {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:47 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:47.015Z INFO [input.s3] s3/collector.go:196 Message visibility timeout updated to 300 seconds {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:47 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:47.015Z INFO [input.s3] s3/collector.go:196 Message visibility timeout updated to 300 seconds {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:47 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:47.016Z INFO [input.s3] s3/collector.go:196 Message visibility timeout updated to 300 seconds {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:47 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:47.016Z INFO [input.s3] s3/collector.go:196 Message visibility timeout updated to 300 seconds {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:47 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:47.022Z INFO [input.s3] s3/collector.go:196 Message visibility timeout updated to 300 seconds {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:47 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:47.022Z INFO [input.s3] s3/collector.go:196 Message visibility timeout updated to 300 seconds {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:47 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:47.022Z INFO [input.s3] s3/collector.go:196 Message visibility timeout updated to 300 seconds {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:47 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:47.022Z INFO [input.s3] s3/collector.go:196 Message visibility timeout updated to 300 seconds {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:47 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:47.025Z INFO [input.s3] s3/collector.go:196 Message visibility timeout updated to 300 seconds {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:47 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:47.026Z INFO [input.s3] s3/collector.go:196 Message visibility timeout updated to 300 seconds {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:47 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:47.026Z INFO [input.s3] s3/collector.go:196 Message visibility timeout updated to 300 seconds {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:47 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:47.026Z INFO [input.s3] s3/collector.go:196 Message visibility timeout updated to 300 seconds {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:47 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:47.026Z INFO [input.s3] s3/collector.go:196 Message visibility timeout updated to 300 seconds {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:47 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:47.026Z INFO [input.s3] s3/collector.go:196 Message visibility timeout updated to 300 seconds {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:47 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:47.026Z INFO [input.s3] s3/collector.go:196 Message visibility timeout updated to 300 seconds {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:47 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:47.031Z INFO [input.s3] s3/collector.go:196 Message visibility timeout updated to 300 seconds {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:47 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:47.031Z INFO [input.s3] s3/collector.go:196 Message visibility timeout updated to 300 seconds {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:47 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:47.032Z INFO [input.s3] s3/collector.go:196 Message visibility timeout updated to 300 seconds {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}
Jun 15 11:56:47 ip-172-31-26-170 filebeat[368069]: 2021-06-15T11:56:47.034Z INFO [input.s3] s3/collector.go:196 Message visibility timeout updated to 300 seconds {"queue_url": "https://my_queeue_url", "region": "ap-south-1"}

```

is there any workaround for this? Need help ASAP, facing this in the production environment 😐

---

<div class="post-metadata">

### Author: ![ChrsMark](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrsmark/32/55858_2.png) [@ChrsMark](https://discuss.elastic.co/u/ChrsMark)
#### Post date: [June 15, 2021, 1:38pm UTC](https://discuss.elastic.co/t/filebeat-stops-processing-logs-for-aws-module/275993/2 "2021-06-15T13:38:29Z")

</div>

Hi!

What version do you use? Also could you provide full configuration for reference?

@Kaiyan_Sheng could you please help with this one?

---

<div class="post-metadata">

### Author: ![ankitdevnalkar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ankitdevnalkar/32/46158_2.png) [@ankitdevnalkar](https://discuss.elastic.co/u/ankitdevnalkar)
#### Post date: [June 15, 2021, 3:10pm UTC](https://discuss.elastic.co/t/filebeat-stops-processing-logs-for-aws-module/275993/3 "2021-06-15T15:10:03Z")

</div>

Hey @ChrsMark Thanks for the quick response. I have resolved it by restarting the server where Filebeat is running. I am suspecting its issue with high CPU usage.

---

<div class="post-metadata">

### Author: ![Kaiyan\_Sheng](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kaiyan_sheng/32/38247_2.png) [@Kaiyan\_Sheng](https://discuss.elastic.co/u/Kaiyan_Sheng)
#### Post date: [June 15, 2021, 3:42pm UTC](https://discuss.elastic.co/t/filebeat-stops-processing-logs-for-aws-module/275993/4 "2021-06-15T15:42:31Z")

</div>

Hi! What version of Filebeat are you running? There was a major bug in Filebeat aws module and was fixed by [[Filebeat] aws-s3 input visibility timeout extensions lead to deadlocked ACKHandlers · Issue #25750 · elastic/beats · GitHub](https://github.com/elastic/beats/issues/25750) and backported to 7.13. If it's possible, could you give 7.13.2 a try, thanks!!

---

<div class="post-metadata">

### Author: ![ankitdevnalkar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ankitdevnalkar/32/46158_2.png) [@ankitdevnalkar](https://discuss.elastic.co/u/ankitdevnalkar)
#### Post date: [June 15, 2021, 5:11pm UTC](https://discuss.elastic.co/t/filebeat-stops-processing-logs-for-aws-module/275993/5 "2021-06-15T17:11:00Z")

</div>

Hey @Kaiyan_Sheng, Thanks for the response. I am using 7.10.1

---

<div class="post-metadata">

### Author: ![Kaiyan\_Sheng](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kaiyan_sheng/32/38247_2.png) [@Kaiyan\_Sheng](https://discuss.elastic.co/u/Kaiyan_Sheng)
#### Post date: [June 15, 2021, 5:34pm UTC](https://discuss.elastic.co/t/filebeat-stops-processing-logs-for-aws-module/275993/6 "2021-06-15T17:34:15Z")

</div>

OK! Could you give 7.13.2 a try?

---

<div class="post-metadata">

### Author: ![ankitdevnalkar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ankitdevnalkar/32/46158_2.png) [@ankitdevnalkar](https://discuss.elastic.co/u/ankitdevnalkar)
#### Post date: [June 15, 2021, 5:42pm UTC](https://discuss.elastic.co/t/filebeat-stops-processing-logs-for-aws-module/275993/7 "2021-06-15T17:42:34Z")

</div>

Would it be compatible with Elasticsearch and Kibana 7.10

---

<div class="post-metadata">

### Author: ![Kaiyan\_Sheng](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kaiyan_sheng/32/38247_2.png) [@Kaiyan\_Sheng](https://discuss.elastic.co/u/Kaiyan_Sheng)
#### Post date: [June 15, 2021, 7:38pm UTC](https://discuss.elastic.co/t/filebeat-stops-processing-logs-for-aws-module/275993/8 "2021-06-15T19:38:56Z")

</div>

It should be based on the [compatibility matrix](https://www.elastic.co/support/matrix#matrix_compatibility) 🙂

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 13, 2021, 9:39pm UTC](https://discuss.elastic.co/t/filebeat-stops-processing-logs-for-aws-module/275993/9 "2021-07-13T21:39:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
