APM configuration for AWS

[hogbinj (James Hogbin)]

This is a schematic of my AWS app setup

I have an ECS auto scale cluster for my front end based on Django which retrieves data from Postgres and ElasticSearch. I have bunch of Lambdas either event driven or schedule driven with some Step function orchestration for long running or linear processes.

Everything in AWS logs to CloudWatch who's logs I can ship to ES using function beat.

However I;'m struggling with any form of correlation in APM.

I'd like to look at the underlying component performance to work out bottlenecks in the app.

e.g. is my UI sluggish because my cluster is short on resources or in the the middle of autoscaling
or is my data slow because my lambda data ingest is hammering the ES cluster etc etc

How best should I configure the AWS environment to feed APM.

Do I need speared file beat/metric beats on each ECS instance?
Do I need multiple function beats one for ECS, one for EventBridge Lambdas one for step function lambdas with custom index names?

APM is OK ish for performance tuning ES queries but I'm struggling to configure what is a fairly simple use case in APM

[basepi (Colton Myers)]

Part of the problem is that we don't currently have support for AWS Lambda, which certainly creates a hole in your observability story in AWS. It's something that's on our roadmap, but we don't know when we'll be able to deliver it.

I have to admit that beyond that my AWS-foo is a bit lacking. It seems like you'll definitely want a place to run Metricbeat with the AWS module so that you can get metrics on a lot of those AWS services. And yes, I'd install metricbeat and filebeat on everything that's supported.

Correlation is an ever-present problem that we're always trying to improve. Once you have all the relevant data in Elasticsearch, we try to surface things automatically, but often you have to use the high level views to get a narrow time band for the issue and then start inspecting the collected logs and metrics in that window.

Sorry I'm not more help here, perhaps others will chime in. And hopefully we can get Lambda support into Elastic APM soon!