Elastic APM incomplete data collection

Elastic Observability APM
1

We have setup Single APM Server for collecting events from js-agent and java agent. But, we faced some issues after deployment of APM Server and agent.

APM Server Configuration:
Version: 7.16.2
apm-server.auth.anonymous.rate_limit.event_limit: 15000
apm-server.auth.anonymous.rate_limit.ip_limit: 2000
output.elasticsearch.bulk_max_size: 1000
queue.mem.events: 150000
Instance JVM: 4GB
Instance vCPU: 2

ES Configuration:
master node: 1
data node: 2

Kibana version: 7.16.2

Elasticsearch version: 7.16.2

APM Server version: 7.16.2

APM Agent language and version: js agent

Browser version: Chrome and Firefox

Original install method (e.g. download page, yum, deb, from source, etc.) and version: rpm

Fresh install or upgraded from other version?: Fresh install

Expectation:

  • All the page-load events must be stored in Elasticsearch.

Issue:

  • Found some events were not available in Elasticsearch

Could you please provide your valuable support and insight?

2

Hi @Suresh_Ghatuwa,

Thanks for reaching out!

Could you confirm the RUM agent is sending the events to the APM server?

Prerequisite (because of step 4): Troubleshooting | APM Real User Monitoring JavaScript Agent Reference [5.x] | Elastic

  1. Open the browser dev tools
  2. Check the network tab
  3. Look for "events/".
  4. You should see the associated requests and the payload (where you can see the transaction.type page-load).

Thanks,
Alberto

3

Yes RUM is sending data to APM Server. But when checking from kibana, we didnot found data passed from js agent or java agent.

And this is random behaviour, some page, it is fine but for most of the pages, data not stored in ES.

We also enabled logs in APM Server and found not modified message. Does this means data will not be stored in ES?