We have observed that the user agent string (user_agent.original field name) in kibana logs is showing as
for all or any web view browsers instead of the actual user agent string.
To verify and confirm it, we have started sending the user agent string in tags of the APM transactional RUM events at client side to verify the same which we did (please refer the below screenshot).
Please help us in resolving this issue which we are facing!
Can you confirm this data is coming from the an APM agent, if so, which one?
@tylersmalley, Yes it is coming from APM agent after we added the below configuration
- pipeline: "apm_user_agent"`
I am going to move this to the APM group, someone there should be able to provide some more help
This indeed looks suspicious. There are currently two ways how the APM Server enriches data with the UserAgent information: It either parses the information directly from the
User-Agent header from the agent request, or in case the ndjson body sent by the agent includes a
User-Agent in the
context.request.headers, this information is set as
You mention you added the
UserAgent as label. Where did you parse the information from, is it sent as headers with the agent request?
@simitt thank you for looking into this. With reference to the screenshot above in the question, the labels.userAgent value is being sent from the browser's user agent from the client side by adding it in the tags during the APM RUM transaction events. Let me know if more information is required from my side.
The APM Server doesn't process this information from
labels, but either from the
headers that are sent by the agent, or from whatever is sent within
user-agent. You should be able to find the headers in your APM Server logs. Can you confirm that the user agent information there is the same as the one you are sending in the
Which versions of Elasticsearch, APM Server and the RUM agent are you running?
With references to below screenshot, the APM server responds with
fp/f99c5a in it's header of server key name while in the request header of User-Agent the value is correct. I am just sending the actual user agent string (same as User-Agent request header) in tags for other purposes because due to APM server or Kibana logs, the user_agent.original field name is incorrect.
Please note that we are using
Unfortunately I cannot reproduce this behavior locally, sending requests with some of the
user-agent info showed in your examples, results in properly set
user_agent information in Elasticsearch.
Do you have anything additionally set up, any proxies in between or any non-default configurations that could give a hint? And can you share which versions of the agent, APM Server and Elasticsearch you are using?
No proxy setup.
v7.1.1 is the deployment version.
The only configuration change is at in User Settings Overrides (apm-server.yml) of APM server which is
- pipeline: "apm_user_agent"
The issue appears when the
user-agent string contains
 chars. I have forwarded the issue internally and will keep you updated.
Thank you, @simitt for your continuous support.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.