Collect logs from AWS Fargate application to Elastic Cloud

Elastic Observability
1

I have an application running on AWS Fargate. I have already added the Elastic APM agent to the application so I get metrics, transactions etc. but no log yet.

So how do I collect logs from an AWS Fargate application to Elastic Cloud?

I found the AWS Fargate integration but that seems to be mostly for collecting Fargate-specific metrics, not logs.
There is also the Filebeat AWS Fargate integration which seems to be about collecting Fargate application logs so that's good. It contains an example config:

- module: awsfargate
  log:
    enabled: true

Where do I deploy this config? Sorry if this is a newbie question, I'm just getting started with Elastic Cloud.

1 Like
2

Hi!

I'm also interested in this feature.
How did you go about this? Did you resolve this for use case? If so, mind sharing how you did it?

Best regards

3

I believe you need to deploy a separate Filebeat agent (somewhere on AWS) with the given config. But for various reasons I will fetch the logs from Cloudwatch instead, for which there are also Filebeat and Elastic Agent integrations.

1 Like
4

Okey, thank you for the reply. I am also currently going with Filebeat as side car. Works decently.

5

I think you could also do something like

Fargate Logs -> Firelens -> Kinesis -> Elasting Agent or Filebeat or Elastic Serverles Forwarder -> Elasticsearch

6

I've tried the Firelens approach as well but then they don't show up in CloudWatch logs anymore but I want to keep them in CloudWatch as well just in case the log shipping pipeline fails.