APM Java agent - Implement client-side load balancing

Kibana version: 7.3.0

Elasticsearch version: 7.3.0

APM Server version: 7.3.0

APM Agent language and version: 1.9.0

Hi,

I am trying to implement client side load-balance for multiple apm servers. I am using the below configuration -

server_urls=http://loadbalancer/<apm_server-1>:8200,http://loadbalancer/<apm_server-2>:8200,http://loadbalancer/<apm_server-2>:8200

But, it is giving me the below error.

2019-08-27 13:55:24.806 [apm-reporter] ERROR co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Error sending data to APM server: Server returned HTTP response code: 502 for URL: http://loadbalancer/<apm_server>:8200/intake/v2/events, response code is 502

Could you please help here.

Thanks.

Regards,
Rijash

Could you send me the output of curl http://loadbalancer/<apm_server>:8200/ and curl -H 'Content-Type: application/x-ndjson' -XPOST http://loadbalancer/<apm_server>:8200/intake/v2/events?

Thanks for your response felix.

PFB the output

:~ # curl -H 'Content-Type: application/x-ndjson' -XPOST http://loadbalancer/<apm_server>:8200/intake/v2/events?
curl: (6) Could not resolve host: loadbalancer
:~ # curl http://loadbalancer/<apm_server>:8200/
curl: (6) Could not resolve host: loadbalancer

Seems like there are connection issues so that the agent can’t connect to your load balancer. Once those are resolved, the agent should be able to connect.

Question: it seems you already have a load balancer. Why do you also want to apply client side load balancing?

My intention is to route the events uniformly onto a list of apm servers configured for the environment.

I think below configuration is working for the same in apm java agent.

server_urls=http://<apm_server_1>:8200,http://<apm_server_2>:8200,http://<apm_server_3>:8200

My next question is, can it be work for RUM agent configuration as well.

Like is the below configuration is correct for RUM agent configuration,

      var pageName = window.location.pathname;
      window.elasticApm.init({
      serviceName: 'bizx',
      environment: 'preview',
      pageLoadTransactionName: pageName,
      logLevel: 'info',
      serverUrl: 'http://<apm_server_1>:8200,http://<apm_server_2>:8200,http://<apm_server_3>:8200'
              });

The RUM agent currently does not support that. Could you create a feature request issue in the rum agent’s GitHub repo?

Sure.

But in that case, if i want to handle the load for RUM agent data. How can we achieve it ?

Like how can we distribute the RUM data from multiple host to multiple APM servers?

Is there any option available ?

Currently only by placing an HTTP load balancer in front of your APM Servers.