Custom Pipeline with Elastic Agent AWS EC2 Cloudwatch Log integration

I'm trying to migrate some Cloudwatch logs currently being captured by Functionbeat to a fleet-managed Elastic Agent integration (AWS EC2 Cloudwatch). The logs are being captured, as expected. However, I am getting a grok error from the logs-aws.ec2_log pipeline. These are custom log formats, and I currently parse them with a custom pipeline. However, there doesn't seem to be an option to do this with this integration. The error forces the logs-aws.ec2_log pipeline to exit before logs-aws.ec2_log@custom can be called, and while the integration will let me specify additional processors, it won't allow me to override the pipeline setting, like I'm able to with the Custom Log integration.

Help?

I think I may have determined the issue. I'm pretty sure I selected the wrong integration - selected Cloudwatch, should've been Cloudtrail. :rofl:

I'll update and keep you posted.

2 Likes

The beta AWS Custom Logs integration resolved the issue.