Kibana version: 7.1.1 oss (AWS Elasticsearch Service)
Elasticsearch version: 7.1.1 oss (AWS Elasticsearch Service)
APM Server version: 7.1.1 oss
APM Agent language and version: Ruby/Rails 7.1.1
Our application is a rails REST api app running in AWS elastic beanstalk, there are no static assets served, and no endpoints that don't ultimately call the rails app.
The application gets about 50k HTTPS requests per minute per the EB monitoring console.
In APM we're only seeing about 10k requests per minute ending up logged as apm transactions.
Is there anything obvious that I should be checking, or is there a reason why this kind of thing might be obviously different? I'm pretty new to this kind of stack, so I'm not sure what the discrepancy might be from.
EDIT: I have a load balancer pointed at multiple APM servers, and the agents are pointed at that load balancer. I found that if I scale up the number of apm servers, the number of transactions successfully reaching the ES server increases, though it's not quite linearly. The APM servers don't have very high CPU usage, there isn't a max number of simultaneous requests, and I can't think of any other bottleneck, but nevertheless, increasing the number of APM servers does seem to allow more transactions to successfully make it over. Does this ring a bell for anyone?