APM RUM Client IP not correct

Elastic Observability APM
1

Kibana version: 7.12.1

Elasticsearch version: 7.12.1

APM Server version: 7.12.1

APM Agent language and version: Java agent (elastic-apm-rum.umd-5.6.1.js)

Browser version: different browsers

Original install method (e.g. download page, yum, deb, from source, etc.) and version: deb

Fresh install or upgraded from other version? Upgraded regularely

Is there anything special in your setup? For example, are you using the Logstash or Kafka outputs? Are you using a load balancer in front of the APM Servers? Have you changed index pattern, generated custom templates, changed agent configuration etc.
We have instrumented the Java Agents in the backend of the application and also added RUM in the frontend. We see data from both (backend and frontend)

Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):
APM in Kibana shows at client IP always the internal IP of the Java Agent, not the real IP from the end user who is tracked by apm rum.
Therefore, no real breakdown of User Experience > " Page load duration by region" is possible.
"Page load duration by region" is empty.

Steps to reproduce:

  1. Fire RUM Events in browser

Errors in browser console (if relevant): no errors in browser console

Provide logs and/or server output (if relevant): no logs regarding issue

2

Maybe I got some more insight to this, it looks like we don't see the correct IP as it is behind a WAF (Web Application firewall) ...
The real client IP is in a request header.

In Logstash I am able to map this request header to a specific field.
Is there any possibility to use this request header information in APM - Server, so that it is used as client.ip?

Thanks

3

@binschlag what is the header? The apm-server should pick the client IP out of various standard headers: Forwarded, X-Real-IP, X-Forwarded-For.

4

@axw The header is called

X-{WAF}-Forward

, so a name defined by us with the name of the Web Application Firewall in it (here with placeholder {WAF}). So I guess therefore APM server is not picking this, but the question is if i can configure APM-Server to use a specific header

5

Sorry for the late reply.

So I guess therefore APM server is not picking this, but the question is if i can configure APM-Server to use a specific header

Right, it's not picking that up. There's currently no way to configure APM Server to use another header for this. We could add configuration, but I'd like to know more about the problem first:

  1. Which WAF are you using?
  2. Is there an option to use a more standard header instead?
  3. Is there some way we can reasonably automatically detect these headers? (Probaby depends on answer to #1.)
6

Thanks for your answer, that this is not possible with APM server in the moment.

We will check to reconfigure WAF or put a proxy or something before the APM server to forward the client IP in a standard header, should be able to solve it that way.

1 Like
7

@axw we reconfigured our environment to forward the client IP in a header which is picked up by APM server and this works now. So no need to add configuration or something for us.

Thanks for your help.

1 Like
8

Glad to hear it! Thanks for letting us know.

9

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.