APM Agent language and version : Java 1.25.0 & 1.21.0
Description of the problem including expected versus actual behavior. Please include screenshots (if relevant) :
I use APM server 6.8.11 to collect several servers apm data.
Webflux + APM Agent 1.25.0:
The second pic shows that headers' key are all in lowercase. These mixed keys make the same header store different field in elasticsearch. The original headers are just like the first pic.
So why the second pic shows headers' key in lowercase and is there any way to make it standard?
Native support for WebFlux was only added in 1.21.0, so this may be the reason, although I am not sure why headers were captured with lowercase before.
I assume the screenshots above describe different transaction groups. What was the full name of the transaction captured in 1.21.0 and what is the name of that captured by 1.25.0?
Thank you for reply!
The screenshots above describe the same transaction groups. What I did is change from webmvc to webflux and change agent version from 1.21.0 to 1.25.0, so the difference in screenshots above took place. (name of the transaction is eg. UserQueryController#queryUserByConditions).
This makes me in trouble that I set headers "dynamic":true in Elasticsearch apm index template, but it can only fit one when lowercase and uppercase are both exist. Is there any solution? By the way, I use apm server 7.14.0 in test environment, apm agent 1.21.0 and 1.25.0 are both store in right way just like the first screenshot, so I doubt that the problem is related to apm server version?
Ahh, in this case the difference is probably related to the framework change, rather than the agent change. The agent only reads headers from the framework and serializes it within the span/transaction context, without applying toLowerCase or anything of this sort.
I don't know why you had to apply special mapping for the apm indices, but ideally HTTP header fields should reflect the fact that HTTP header names are case-insensitive, which means different frameworks may produce different letter case. So I believe the best solution is fix the field mapping.
Otherwise, if that is not possible, you may setup an ingest pipeline to normalize all header names (e.g. to lowercase).
Thank you for reply and suggestion!
Now I still can not combine these two version of letter case, but I can search them individually. Although they won't store in elastic in the same field, it just works.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
