# RUM JS not collecting user agent location **URL:** \ **Category:** User Experience\ **Created:** [July 29, 2022, 2:26pm UTC](https://discuss.elastic.co/t/rum-js-not-collecting-user-agent-location/310993 "2022-07-29T14:26:08Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![marone](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marone/32/87145_2.png) [@marone](https://discuss.elastic.co/u/marone)\ **Post date:** [July 29, 2022, 2:26pm UTC](https://discuss.elastic.co/t/rum-js-not-collecting-user-agent-location/310993/1 "2022-07-29T14:26:08Z")
This is my 3rd time configuring elastic APM, and always face a problem populating the "Page load distribution" over countries. By default elastic uses geoip processor to add client location for rum data but it's not the case for me. There is no index called `traces-apm.rum-default` created for RUM data. Checking [https://demo.elastic.co](https://demo.elastic.co) User experience dashboard indeed have the dashboard populated with data and the `traces-apm.rum-default` index exists. I configured RUM for a Django application, I added to the base template the following: ```auto ``` I am using elastic 7.15. What am I missing? I checked the [doc for rum](https://www.elastic.co/guide/en/apm/agent/rum-js/current/intro.html) many times but could not find how to include user agent location for rum data, and could not figure out why I don't have an index like `traces-apm.rum-default`? The error I'm having: ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/b/fb1cb590ca2e9831059b88c1d5966a0961d2789a.png) Thanks. ---
**Author:** ![Alberto\_Delgado](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alberto_delgado/32/98488_2.png) [@Alberto\_Delgado](https://discuss.elastic.co/u/Alberto_Delgado)\ **Post date:** [August 2, 2022, 3:16pm UTC](https://discuss.elastic.co/t/rum-js-not-collecting-user-agent-location/310993/2 "2022-08-02T15:16:55Z")
Hi @marone, Thanks for reaching out and apologies for the delay. RUM doesn't handle anything related to ip address and geo information. That is something that APM server [handles](https://www.elastic.co/guide/en/apm/guide/current/ingest-pipelines.html). That's why you were not finding information on the RUM documentation. -- Please, being in the devtools console page execute the following query: `GET /_ingest/geoip/stats` If the output you are getting is something like this: ![Screenshot 2022-08-02 at 16.54.06](https://us1.discourse-cdn.com/elastic/original/3X/4/c/4c6abde2e533bbe373521678258e4ff40edf88bb.png) It means that there has been some issue while downloading the **geoip** database. So, what you need to do is to "turn it off and on again" the geo pipeline which will force the download of the **geoip** database How to turn it off: ```auto PUT /_cluster/settings?pretty { "persistent" : { "ingest.geoip.downloader.enabled" : false } } ``` How to turn it on: ```auto PUT /_cluster/settings?pretty { "persistent" : { "ingest.geoip.downloader.enabled" : true } } ``` After turning it on you will see Elasticsearch logs like this ones: ![Screenshot 2022-08-02 at 16.55.06](https://us1.discourse-cdn.com/elastic/original/3X/1/f/1f42e966a6fc23486a493ec98b8e7198011532e3.png) If you execute the query `GET /_ingest/geoip/stats` again you will see that now the database has been downloaded properly: ![Screenshot 2022-08-02 at 16.55.45](https://us1.discourse-cdn.com/elastic/original/3X/4/9/497b35f1b8bc89f0567788aafab32f830d60b727.png) After doing this, if you want to get rid of the "unable to perform term join" issue, what you need to do is to visit your webpage to generate new events, then the User Experience Page will show the data properly, for instance: ![Screenshot 2022-08-02 at 17.10.28](https://us1.discourse-cdn.com/elastic/original/3X/f/d/fdb1027fafd23f386d40d36698266e06e0e196e4.png) Thanks, Alberto ---
**Author:** ![marone](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marone/32/87145_2.png) [@marone](https://discuss.elastic.co/u/marone)\ **Post date:** [August 2, 2022, 3:38pm UTC](https://discuss.elastic.co/t/rum-js-not-collecting-user-agent-location/310993/3 "2022-08-02T15:38:00Z")
> [@Alberto\_Delgado](#): > > ```auto > PUT /_cluster/settings?pretty > { > "persistent" : { > "ingest.geoip.downloader.enabled" : true > } > } > > ``` Thank you so much @Alberto_Delgado for the precise answer. When doing the steps you described indeed I didn't find the **geoip** database, once I turn it off/on it is downloaded, and didn't get any geo data after, I believe because I tested it on localhost right? I need to deploy it ASAP and get back toyou if needed 😄. Marwane. ---
**Author:** ![marone](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marone/32/87145_2.png) [@marone](https://discuss.elastic.co/u/marone)\ **Post date:** [August 25, 2022, 10:00am UTC](https://discuss.elastic.co/t/rum-js-not-collecting-user-agent-location/310993/4 "2022-08-25T10:00:10Z")
Hey again @Alberto_Delgado, First sorry for the long time it took. I deployed an [application](https://www.elastic.co/blog/performing-real-user-monitoring-rum-with-elastic-apm) internally in our servers and tried to access it via the public url but still the map empty ☹ I checked also with `GET /_ingest/geoip/stats` to make sure that there are some database, and did the steps again but still have the error message `unable to perform term join`. What could be wrong please? ---
**Author:** ![kyungeunni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kyungeunni/32/125539_2.png) [@kyungeunni](https://discuss.elastic.co/u/kyungeunni)\ **Post date:** [August 26, 2022, 3:42pm UTC](https://discuss.elastic.co/t/rum-js-not-collecting-user-agent-location/310993/5 "2022-08-26T15:42:43Z")
Hi @marone, I'm so sorry that the issue is not solved. At this point we should check if the relevant fields are generated in the documents. Could you please run this query in Dev Tools: ```auto GET apm*/_search { "query": { "bool": { "filter": [ { "exists": { "field": "transaction.experience" } }, { "exists": { "field": "client.geo.country_iso_code" } } ] } }, "_source": ["client.geo.country_iso_code"] } ``` This query is to see if the User Experience documents contain the geo location data. Can you see any hits (`hits.total.value > 0`)? If you can, data required for the Page load distribution map are generated by the geoip processor correctly and we need to investigate this issue further. If you don't see any hits with the above query, we can try to debug by setting a custom header `X-Forwarded-For`(it is useful if you are running things locally to debug). When initiating apm, try adding this option: ```auto elasticApm.init({ .... apmRequest: ({ xhr }) => { xhr.setRequestHeader('X-Forwarded-For', ) return true; } }) ``` In this case, the ingest process should add `client.geo.country_iso_code` after looking up the location of IP when storing the documents. Try visiting your website and running the first query again, you should get hits. Then the User Experience Page should show the data properly. If any of these doesn't work out, please let us know Happy to investigate it further. Thanks, Kyungeun ---
**Author:** ![marone](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marone/32/87145_2.png) [@marone](https://discuss.elastic.co/u/marone)\ **Post date:** [August 30, 2022, 1:19pm UTC](https://discuss.elastic.co/t/rum-js-not-collecting-user-agent-location/310993/6 "2022-08-30T13:19:15Z")
Thank you @kyungeunni for your time. I tried the above code and still no `client.geo.country_iso_code` field in the `apm*` index, this is what I did: 1. retrieved my public ipv4 using [https://whatismyipaddress.com/](https://whatismyipaddress.com/) 2. updated the rum code as follow: ```auto import { init as initApm } from '@elastic/apm-rum' var apm = initApm({ // Set required service name (allowed characters: a-z, A-Z, 0-9, -, _, and space) serviceName: 'carfront', // Set the version of your application // Used on the APM Server to find the right sourcemap serviceVersion: '0.90', environment: 'demo', secretToken: '', // Set custom APM Server URL (default: http://localhost:8200) serverUrl: 'http://localhost:8200', // For distributed tracing to different origin (CORS) distributedTracingOrigins: ['http://cardatabase:8080', 'http://localhost:8080'], apmRequest: ({ xhr }) => { xhr.setRequestHeader('X-Forwarded-For', 'xxx.xxx.xxx.xxx') return true; } }) export default apm; ``` 1. started the server but showing no errors when opening chrome dev tools, the request made by apm has no header 'X-Forwarded-For': ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/f/1f4012f36105fe6892f2633f03a3bbe70071b396.png) What I did is it correct? Thanks. ---
**Author:** ![kyungeunni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kyungeunni/32/125539_2.png) [@kyungeunni](https://discuss.elastic.co/u/kyungeunni)\ **Post date:** [August 30, 2022, 2:26pm UTC](https://discuss.elastic.co/t/rum-js-not-collecting-user-agent-location/310993/7 "2022-08-30T14:26:06Z")
Hi @marone, Yes, you should see the `X-forwared-header` in the request header if configured correctly: ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/3/436eeedab951e0eaaafea310b131b49626a4055c.png) Your configuration looks correct (apart from the missing `,` right before `apmRequest: ({ xhr }) => {` line, but I'm sure it's a typo). I'm wondering if it's cache related issue(either the requests or the javascript file). Could you try again after disabling cache on your browser? ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [August 30, 2022, 2:37pm UTC](https://discuss.elastic.co/t/rum-js-not-collecting-user-agent-location/310993/8 "2022-08-30T14:37:07Z")
Hi @kyungeunni Question: Are you testing this only on internal browsers / IP addresses if so there will no geo ip information as internal IP address can not be looked up / converted to geo data. ---
**Author:** ![kyungeunni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kyungeunni/32/125539_2.png) [@kyungeunni](https://discuss.elastic.co/u/kyungeunni)\ **Post date:** [August 30, 2022, 2:41pm UTC](https://discuss.elastic.co/t/rum-js-not-collecting-user-agent-location/310993/9 "2022-08-30T14:41:52Z")
Hi @stephenb No, I'm testing this with public IPs, also @marone seems to be on the same page: > 1. retrieved my public ipv4 using [https://whatismyipaddress.com/](https://whatismyipaddress.com/) ---
**Author:** ![marone](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marone/32/87145_2.png) [@marone](https://discuss.elastic.co/u/marone)\ **Post date:** [August 30, 2022, 2:43pm UTC](https://discuss.elastic.co/t/rum-js-not-collecting-user-agent-location/310993/10 "2022-08-30T14:43:26Z")
> [@kyungeunni](#): > > Your configuration looks correct (apart from the missing `,` right before `apmRequest: ({ xhr }) => {` line, but I'm sure it's a typo). Yes was a typo only when I did copy/paste here when answering you. > [@kyungeunni](#): > > I'm wondering if it's cache related issue(either the requests or the javascript file). Could you try again after disabling cache on your browser? I disabled cache, tried incognito mode and changed the web browser to Firefox and still not working. ---
**Author:** ![marone](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marone/32/87145_2.png) [@marone](https://discuss.elastic.co/u/marone)\ **Post date:** [August 30, 2022, 2:48pm UTC](https://discuss.elastic.co/t/rum-js-not-collecting-user-agent-location/310993/11 "2022-08-30T14:48:59Z")
One more detail, the application is running using docker on local on windows. is it related maybe? ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [August 30, 2022, 2:50pm UTC](https://discuss.elastic.co/t/rum-js-not-collecting-user-agent-location/310993/12 "2022-08-30T14:50:37Z")
> [@kyungeunni](#): > > No, I'm testing this with public IPs, also @marone seems to be on the same page: > > > 1. retrieved my public ipv4 using [https://whatismyipaddress.com/](https://whatismyipaddress.com/) But I see all this `serverUrl: 'http://localhost:8200',` That tells me your browser and the APM server are running on the same host... so that would not use your public IP. Do you see your public IP in the RUM data? Just double checking... I have seen people spend a lot of time on very similar ---
**Author:** ![marone](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marone/32/87145_2.png) [@marone](https://discuss.elastic.co/u/marone)\ **Post date:** [August 30, 2022, 2:52pm UTC](https://discuss.elastic.co/t/rum-js-not-collecting-user-agent-location/310993/13 "2022-08-30T14:52:47Z")
> [@marone](#): > > ```auto > apmRequest: ({ xhr }) => { > xhr.setRequestHeader('X-Forwarded-For', 'xxx.xxx.xxx.xxx') > return true; > > ``` Hey @stephenb, I've added the above code and 'xxx.xxx.xxx.xxx' is my public ip to debug ---
**Author:** ![kyungeunni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kyungeunni/32/125539_2.png) [@kyungeunni](https://discuss.elastic.co/u/kyungeunni)\ **Post date:** [August 30, 2022, 3:05pm UTC](https://discuss.elastic.co/t/rum-js-not-collecting-user-agent-location/310993/14 "2022-08-30T15:05:41Z")
> That tells me your browser and the APM server are running on the same host... Yes, that's why I've suggested setting `X-Forwarded-For`. When APM Server receives the request and `X-Forwarded-For` header exists, its value will be used to determine the client's GEO location, according to here: [Anonymous authentication | APM User Guide [8.11] | Elastic](https://www.elastic.co/guide/en/apm/guide/current/anonymous-auth.html#derive-client-ip) What I'm doing here is basically taking advantage of this behaviour of APM server, I'm providing the public IP to `X-Forwarded-For` header so that APM server will look up the geo location of this public IP, rather than my internal IP, if that makes sense.(It is a hack to test geo ingest pipeline locally.) I'll attach some screenshots from my local tests: - Request sent: ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/9/096fbbcdc14abe9f1c7a11f891751bc466223405.png) - RUM document from the request: ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/e/1eefc333c605887c112c79435f6290c83e2065ba.png) ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [August 30, 2022, 3:09pm UTC](https://discuss.elastic.co/t/rum-js-not-collecting-user-agent-location/310993/15 "2022-08-30T15:09:17Z")
Can you show a sample document from the RUM Agent? Are you seeing your public ip in the `client.ip` field ![Screen Shot 2022-08-30 at 8.14.59 AM](https://us1.discourse-cdn.com/elastic/original/3X/1/3/1357bbbde41405d4e30b27acad38c6fc3e073c91.png) ---
**Author:** ![simitt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simitt/32/106406_2.png) [@simitt](https://discuss.elastic.co/u/simitt)\ **Post date:** [August 31, 2022, 11:01am UTC](https://discuss.elastic.co/t/rum-js-not-collecting-user-agent-location/310993/16 "2022-08-31T11:01:24Z")
@marone please also enable DEBUG logging (`logging.level: debug`) in the apm-server for a short time. When enabled, it should log the request details from the RUM request. Please share the debug logs for a specific RUM request (and strip away any sensitive information first). It will be especially interesting to see if a `client.ip` is set at this point. ---
**Author:** ![marone](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marone/32/87145_2.png) [@marone](https://discuss.elastic.co/u/marone)\ **Post date:** [August 31, 2022, 2:21pm UTC](https://discuss.elastic.co/t/rum-js-not-collecting-user-agent-location/310993/17 "2022-08-31T14:21:01Z")
Hello and thanks @all for your support. here is a sample document of RUM data: ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/3/930168a3b507e7c811484112afce099e9241be01.png) The problem is that the IP is private, and not taking the public ip provided manually using the `X-Forwarded-For` header. I added the `logging.level: debug` to apm server and following the logs when refreshing the carfront page: [apm server logs - JustPaste.it](https://justpaste.it/6v6sq) (could not paste it here because they are too long) FYI, I'm using ELK and apm server version 7.15.2 ---
**Author:** ![marone](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marone/32/87145_2.png) [@marone](https://discuss.elastic.co/u/marone)\ **Post date:** [September 19, 2022, 1:41pm UTC](https://discuss.elastic.co/t/rum-js-not-collecting-user-agent-location/310993/18 "2022-09-19T13:41:16Z")
Hello again @kyungeunni, @all For some reason after a while it worked 😃 I can see the geo location of the value we used with `X-Forwarded-For` when initializing apm agent: ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/d/6dd486b3024b5b0939df18cd39b3f012281d443c.png) **My question now is:** How can I pass this header dynamically to elastic for all users visiting the webpage? when I did some research on google, found that they use Nginx and they set this header for incoming http request. is it the only way to do it dynamically please? Thank you. ---
**Author:** ![Alberto\_Delgado](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alberto_delgado/32/98488_2.png) [@Alberto\_Delgado](https://discuss.elastic.co/u/Alberto_Delgado)\ **Post date:** [September 20, 2022, 1:58pm UTC](https://discuss.elastic.co/t/rum-js-not-collecting-user-agent-location/310993/19 "2022-09-20T13:58:23Z")
Hi @marone, Glad that worked! 😄 > when I did some research on google, found that they use Nginx and they set this header for incoming http request. Bear in mind this description from [MDN](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-For) about **X-Forwarded-For** "The `X-Forwarded-For` (XFF) header is a de-facto standard header for identifying the originating IP address of a client connecting to a web server through an HTTP proxy or a load balancer. When traffic is intercepted between clients and servers, server access logs contain the IP address of the proxy or load balancer only. To see the original IP address of the client, the `X-Forwarded-For` request header is used." You need to configure your environment to handle the scenario described above, you cannot do such logic from the client side. (excluding the local modification for testing that you made) Apart from that, once you start having visitors you will start seeing their regions properly differentiated. Is the website already accessible over the internet? Please, have a look at this [link](https://discuss.elastic.co/t/private-ip-addresses-are-logged-in-as-client-ip/236531) since might be helpful for you, too. Thanks, Alberto ---
**Author:** ![marone](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marone/32/87145_2.png) [@marone](https://discuss.elastic.co/u/marone)\ **Post date:** [September 20, 2022, 2:28pm UTC](https://discuss.elastic.co/t/rum-js-not-collecting-user-agent-location/310993/20 "2022-09-20T14:28:26Z")
Thanks a lot @Alberto_Delgado, and the website will be accessible only internally (using vpn), and I think I should set `capture_personal_data` to `false` from what I understood. Right? [Next page](https://discuss.elastic.co/t/rum-js-not-collecting-user-agent-location/310993.md?page=2)