Kibana version: 7.5.1
Elasticsearch version: 7.5.1
APM Server version: 7.5.1
APM Agent language and version: Nodejs. "elastic-apm-node": "^3.3.0"
Original install method (e.g. download page, yum, deb, from source, etc.) and version:
Elasticsearch Cluster: yum
APM Server: Docker image - docker.elastic.co/apm/apm-server:7.5.1
Kibana: Docker Image: kibana:7.5.1
Fresh install or upgraded from other version? Fresh install
Is there anything special in your setup? For example, are you using the Logstash or Kafka outputs? Are you using a load balancer in front of the APM Servers? Have you changed index pattern, generated custom templates, changed agent configuration etc.
- Using load balancer in front APM server
- Agent configuration:
require('elastic-apm-node').start({
serviceName: "capping",
serverUrl:${config.apm.protocol}://${config.apm.host}:${config.apm.port},
active: config.apm.enabled,
transactionSampleRate: config.apm.transaction_sample_rate, // set to 0.2
transactionMaxSpans: config.apm.transaction_max_spans, // set to 50
ignoreUrls: config.apm.ignore_urls // ignore problematic routes
});
Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):
We experiencing a higher response time than usual when using APM, we found the reason of the high response time. it is happening because of one of the application routes. when we are excluding that route from being sent to APM server the response time become as usual.
Info about the problematic route:
- Average latency ~26s/m
- 95th Percentile ~64s/m
- About 1 request per minute
Screenshots of the application response time running the same code under same load:
- We already tried to reduce transactionSampleRate to 5% and Reduced transactionMaxSpans to 50, not helped
any idea why that specific route causes to a higher response time of the application?
Thanks,
Vic M.