Added label to APM which has the incorrect index 'type'

[Matthew_Petrie (Matthew)]

Kibana version: 7.5.1
Elasticsearch version: 7.5.1
APM Server version: 7.5.1
APM Agent language and version: Node.JS 3.3.0
Original install method (e.g. download page, yum, deb, from source, etc.) and version: Elastic Cloud

Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):
I have added 2 labels which are integers to a span using .addLabels({ "bucket": 213 }) however the index seems to have set them to the type keyword. I am wanting to use the labels in visualizations however if I use the Area visualization and choose average for the Y-axis the labels do not appear in the fields dropdown (the labels exist in the documents when I view them in discover). I'm not sure if this is related to the fields having the wrong type in the index or not - if this is the issue how can I correct this?

Any help would be greatly appreciated!

Thanks, Matthew

[axw (Andrew Wilkins)]

@Matthew_Petrie, the Node.js agent currently does not support sending numeric label values; instead, they are stringified. I've opened https://github.com/elastic/apm-agent-nodejs/issues/1598 to track adding support.

In the mean time, you could add a filter to the agent to parse the strings back to numbers before sending to the server.

[Matthew_Petrie (Matthew)]

Thanks @axw - hadn't realised the APM filter function ran after the stringification.

As per the GitHub issue, if I put the filter in it will cause issues with the index - any advice on how to resolve this? (I'm pretty new to Elastic)

Thanks for your reply

[axw (Andrew Wilkins)]

Mapping | Elasticsearch Guide [8.11] | Elastic says:

Except for supported mapping parameters, you can’t change the mapping or field type of an existing field. Changing an existing field could invalidate data that’s already indexed.

If you need to change the mapping of a field, create a new index with the correct mapping and reindex your data into that index.

You have a few options:

• Delete your existing docs and mappings. I assume you want to keep data, I'm just listing this for completeness.
• Start using a different name for your labels, so there's no collision with the existing mapping/field type.
• Create a new mapping, and reindex your data. You could either drop the existing stringified label fields, or convert to numbers, using an Ingest node pipeline.

[Matthew_Petrie (Matthew)]

@axw

Create a new mapping, and reindex your data. You could either drop the existing stringified label fields, or convert to numbers, using an Ingest node pipeline.

This sounds great - hadn't realised you could drop certain fields.

Thanks for your time, it's greatly appreciated!