# Custom Pipeline with Elastic Agent AWS EC2 Cloudwatch Log integration

**URL:** <https://discuss.elastic.co/t/custom-pipeline-with-elastic-agent-aws-ec2-cloudwatch-log-integration/315031>\
**Category:** Elastic Agent\
**Tags:** fleet\
**Created:** [September 23, 2022, 3:39pm UTC](https://discuss.elastic.co/t/custom-pipeline-with-elastic-agent-aws-ec2-cloudwatch-log-integration/315031 "2022-09-23T15:39:51Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![DougR](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dougr/32/48095_2.png) [@DougR](https://discuss.elastic.co/u/DougR)\
**Post date:** [September 23, 2022, 3:39pm UTC](https://discuss.elastic.co/t/custom-pipeline-with-elastic-agent-aws-ec2-cloudwatch-log-integration/315031/1 "2022-09-23T15:39:51Z")

</div>

I'm trying to migrate some Cloudwatch logs currently being captured by Functionbeat to a fleet-managed Elastic Agent integration (AWS EC2 Cloudwatch). The logs are being captured, as expected. However, I am getting a grok error from the `logs-aws.ec2_log` pipeline. These are custom log formats, and I currently parse them with a custom pipeline. However, there doesn't seem to be an option to do this with this integration. The error forces the `logs-aws.ec2_log` pipeline to exit before `logs-aws.ec2_log@custom` can be called, and while the integration will let me specify additional processors, it won't allow me to override the pipeline setting, like I'm able to with the Custom Log integration.

Help?

---

<div class="post-metadata">

**Author:** ![DougR](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dougr/32/48095_2.png) [@DougR](https://discuss.elastic.co/u/DougR)\
**Post date:** [September 23, 2022, 4:08pm UTC](https://discuss.elastic.co/t/custom-pipeline-with-elastic-agent-aws-ec2-cloudwatch-log-integration/315031/2 "2022-09-23T16:08:17Z")

</div>

I think I may have determined the issue. I'm pretty sure I selected the wrong integration - selected Cloudwatch, should've been Cloudtrail. 🤣

I'll update and keep you posted.

---

<div class="post-metadata">

**Author:** ![DougR](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dougr/32/48095_2.png) [@DougR](https://discuss.elastic.co/u/DougR)\
**Post date:** [September 28, 2022, 4:00pm UTC](https://discuss.elastic.co/t/custom-pipeline-with-elastic-agent-aws-ec2-cloudwatch-log-integration/315031/3 "2022-09-28T16:00:13Z")

</div>

The beta AWS Custom Logs integration resolved the issue.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 26, 2022, 4:00pm UTC](https://discuss.elastic.co/t/custom-pipeline-with-elastic-agent-aws-ec2-cloudwatch-log-integration/315031/4 "2022-10-26T16:00:27Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.