# Problem ingesting vpc flow logs

**URL:** <https://discuss.elastic.co/t/problem-ingesting-vpc-flow-logs/239172>\
**Category:** Beats\
**Created:** [June 29, 2020, 6:56pm UTC](https://discuss.elastic.co/t/problem-ingesting-vpc-flow-logs/239172 "2020-06-29T18:56:59Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![andywt123](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andywt123/32/65160_2.png) [@andywt123](https://discuss.elastic.co/u/andywt123)\
**Post date:** [June 29, 2020, 6:57pm UTC](https://discuss.elastic.co/t/problem-ingesting-vpc-flow-logs/239172/1 "2020-06-29T18:57:00Z")

</div>

We are using filebeat 7.6.2 to ingest vpc flow logs. IT appears that it is taking longer then 24 hours for the vpc flow logs to be ingested.  
Everything seems to be working fine for the elb logs. The SQS queue still has messages from noon yesterday in it. The number of messages available does fluctuate. Is this a known issue? How can we rectify this?

---

<div class="post-metadata">

**Author:** ![andywt123](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andywt123/32/65160_2.png) [@andywt123](https://discuss.elastic.co/u/andywt123)\
**Post date:** [July 10, 2020, 2:44pm UTC](https://discuss.elastic.co/t/problem-ingesting-vpc-flow-logs/239172/2 "2020-07-10T14:44:12Z")

</div>

Is the aws vpc flow log supported?

---

<div class="post-metadata">

**Author:** ![nikhilv](https://avatars.discourse-cdn.com/v4/letter/n/ce73a5/32.png) [@nikhilv](https://discuss.elastic.co/u/nikhilv)\
**Post date:** [July 22, 2020, 11:27am UTC](https://discuss.elastic.co/t/problem-ingesting-vpc-flow-logs/239172/3 "2020-07-22T11:27:04Z")

</div>

I'm also facing the same issue, i'm getting error like "ERROR [s3] s3/input.go:475 ReadString failed: context deadline exceede" and SQS queue is full , it's unable to process the messages. I have increased visibility timeout to 45min but still getting the error.

---

<div class="post-metadata">

**Author:** ![andywt123](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andywt123/32/65160_2.png) [@andywt123](https://discuss.elastic.co/u/andywt123)\
**Post date:** [July 22, 2020, 2:09pm UTC](https://discuss.elastic.co/t/problem-ingesting-vpc-flow-logs/239172/4 "2020-07-22T14:09:12Z")

</div>

I have been advised to upgrade to the latest filebeats and use IAM roles that should resolve the issue

---

<div class="post-metadata">

**Author:** ![nikhilv](https://avatars.discourse-cdn.com/v4/letter/n/ce73a5/32.png) [@nikhilv](https://discuss.elastic.co/u/nikhilv)\
**Post date:** [July 22, 2020, 3:12pm UTC](https://discuss.elastic.co/t/problem-ingesting-vpc-flow-logs/239172/5 "2020-07-22T15:12:29Z")

</div>

I'm using latest 7.8 version, you're issue resolved? and what is IAM Role configuration?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 19, 2020, 5:12pm UTC](https://discuss.elastic.co/t/problem-ingesting-vpc-flow-logs/239172/6 "2020-08-19T17:12:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.