# Problem with APM after upgrade to 7.10
**URL:** https://discuss.elastic.co/t/problem-with-apm-after-upgrade-to-7-10/274277
**Category:** APM
**Tags:** docker, server
**Created:** [May 28, 2021, 12:47am UTC](https://discuss.elastic.co/t/problem-with-apm-after-upgrade-to-7-10/274277 "2021-05-28T00:47:06Z")
**Posts on this page:** 10
**Page:** 1
### Author:  [@Luis\_Garavatti](https://discuss.elastic.co/u/Luis_Garavatti)
#### Post date: [May 28, 2021, 12:47am UTC](https://discuss.elastic.co/t/problem-with-apm-after-upgrade-to-7-10/274277/1 "2021-05-28T00:47:06Z")
Guys, today my APM is not working on the turn to create a new index.
The error is:
```auto
"log": "2021-05-28T00:00:02.816Z\tWARN\t[elasticsearch]\telasticsearch/client.go:408\tCannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0x4d3f640, ext:63757756799, loc:(*time.Location)(nil)}, Meta:{\"pipeline\":\"apm\"}, Fields:{\"agent\":{\"ephemeral_id\":\"304360b5-5573-4c97-842a-ea5ac7229f21\",\"name\":\"java\",\"version\":\"1.12.0\"},\"ecs\":{\"version\":\"1.6.0\"},\"host\":{\"architecture\":\"amd64\",\"hostname\":\xxxxxxx\",\"ip\":\"10.23.78.129\",\"name\":\"xxxxxxxxxl\",\"os\":{\"platform\":\"Linux\"}},\"jvm\":{\"gc\":{\"count\":16,\"time\":7987}},\"labels\":{\"name\":\"PS MarkSweep\"},\"observer\":{\"ephemeral_id\":\"4ed24683-847f-40c3-b879-a52c91b0396a\",\"hostname\":\"apm-server-667cfd5674-rvpd9\",\"id\":\"xxxxxxxx\",\"type\":\"apm-server\",\"version\":\"7.10.2\",\"version_major\":7},\"process\":{\"pid\":8566,\"title\":\"/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.171-7.b10.37.amzn1.x86_64/jre/bin/java\"},\"processor\":{\"event\":\"metric\",\"name\":\"metric\"},\"service\":{\"environment\":\"production\",\"language\":{\"name\":\"Java\",\"version\":\"1.8.0_171\"},\"name\":\"xxxxx\",\"node\":{\"name\":\"xxxxx\"},\"runtime\":{\"name\":\"Java\",\"version\":\"1.8.0_171\"}}}, Private:interface {}(nil), TimeSeries:false}, Flags:0x1, Cache:publisher.EventCache{m:common.MapStr(nil)}} (status=400): {\"type\":\"mapper_parsing_exception\",\"reason\":\"failed to parse field [labels.name] of type [scaled_float] in document with id 'ZDFFsHkBBnKI80FPTrz4'. Preview of field's value: 'PS MarkSweep'\",\"caused_by\":{\"type\":\"number_format_exception\",\"reason\":\"For input string: \\\"PS MarkSweep\\\"\"}}\n",
```
Pls help meee
---
### Author:  [@Luis\_Garavatti](https://discuss.elastic.co/u/Luis_Garavatti)
#### Post date: [May 28, 2021, 1:22am UTC](https://discuss.elastic.co/t/problem-with-apm-after-upgrade-to-7-10/274277/2 "2021-05-28T01:22:18Z")
The error now is:
```auto
"2021-05-27T01:19:50.825Z\tWARN\t[elasticsearch]\telasticsearch/client.go:408\tCannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0x33058600, ext:63757675182, loc:(*time.Location)(nil)}, Meta:{\"pipeline\":\"apm\"}, Fields:{\"agent\":{\"name\":\"nodejs\",\"version\":\"3.8.0\"},\"container\":{\"id\":\"183a65910409c969b99fbb8f15c0a910757c32f4543a9f859b63117a2b84c1a7\"},\"ecs\":{\"version\":\"1.6.0\"},\"host\":{\"architecture\":\"x64\",\"hostname\":\"183a65910409\",\"ip\":\"33333333\",\"name\":\"183a65910409\",\"os\":{\"platform\":\"linux\"}},\"labels\":{\"env\":\"production\",\"hostname\":\"183a65910409\"},\"observer\":{\"ephemeral_id\":\"4ed24683-847f-40c3-b879-a52c91b0396a\",\"hostname\":\"apm-server-667cfd5674-rvpd9\",\"id\":\"040df035-2207-4e1a-830d-4c33a4413337\",\"type\":\"apm-server\",\"version\":\"7.10.2\",\"version_major\":7},\"process\":{\"args\":[\"/usr/local/bin/node\",\"/app/bin/www\"],\"pid\":7,\"ppid\":1,\"title\":\"node\"},\"processor\":{\"event\":\"metric\",\"name\":\"metric\"},\"service\":{\"environment\":\"production\",\"framework\":{\"name\":\"express\",\"version\":\"4.17.1\"},\"language\":{\"name\":\"javascript\"},\"name\":\"xxxxxxx\",\"node\":{\"name\":\"183a65910409c969b99fbb8f15c0a910757c32f4543a9f859b63117a2b84c1a7\"},\"runtime\":{\"name\":\"node\",\"version\":\"14.15.1\"},\"version\":\"0.0.1\"},\"span\":{\"self_time\":{\"count\":0,\"sum\":{\"us\":0}},\"type\":\"app\"},\"transaction\":{\"name\":\"\",\"type\":\"request\"}}, Private:interface {}(nil), TimeSeries:false}, Flags:0x1, Cache:publisher.EventCache{m:common.MapStr(nil)}} (status=400): {\"type\":\"mapper_parsing_exception\",\"reason\":\"failed to parse field [labels.hostname] of type [scaled_float] in document with id 'O-Noq3kBBnKI80FPAuAe'. Preview of field's value: '183a65910409'\",\"caused_by\":{\"type\":\"number_format_exception\",\"reason\":\"For input string: \\\"183a65910409\\\"\"}}\n",
```
---
### Author:  [@Luis\_Garavatti](https://discuss.elastic.co/u/Luis_Garavatti)
#### Post date: [May 28, 2021, 1:27am UTC](https://discuss.elastic.co/t/problem-with-apm-after-upgrade-to-7-10/274277/3 "2021-05-28T01:27:53Z")
These two errors appear frequently
---
### Author:  [@simitt](https://discuss.elastic.co/u/simitt)
#### Post date: [May 28, 2021, 6:41am UTC](https://discuss.elastic.co/t/problem-with-apm-after-upgrade-to-7-10/274277/4 "2021-05-28T06:41:44Z")
The error indicates that a numeric value has been previously sent and indexed for `labels.hostname`, which creates a numeric mapping for this field. Following documents with a different type for this field (`string` in your example) can not be indexed.
Are you instrumenting multiple different services where you set the `hostname` as label and if so, which APM language agents are you using?
---
### Author:  [@Luis\_Garavatti](https://discuss.elastic.co/u/Luis_Garavatti)
#### Post date: [May 28, 2021, 12:10pm UTC](https://discuss.elastic.co/t/problem-with-apm-after-upgrade-to-7-10/274277/5 "2021-05-28T12:10:51Z")
Hello Silvia,
I'm using nodejs, java and javascript.
How do I solve this?
---
### Author:  [@simitt](https://discuss.elastic.co/u/simitt)
#### Post date: [May 28, 2021, 1:46pm UTC](https://discuss.elastic.co/t/problem-with-apm-after-upgrade-to-7-10/274277/6 "2021-05-28T13:46:05Z")
When instrumenting any service, ensure that the `setLabel` APIs (rumjs - [`apm.AddLabels`](https://www.elastic.co/guide/en/apm/agent/rum-js/5.x/agent-api.html#apm-add-labels), nodejs - [`apm.setLabel`](https://www.elastic.co/guide/en/apm/agent/nodejs/3.x/agent-api.html#apm-set-label) and java - [`transaction setLabel`](https://www.elastic.co/guide/en/apm/agent/java/1.x/public-api.html#api-transaction-add-tag)) use the same value type for the same keys accross all services. For instance, if you set the `hostname` as label key, ensure that the passed in values are always of type `string` or always of a `numeric` type, but do not mix them. The nodejs agent by default stringifies all values, but this can be disabled.
Label fields are indexed dynamically in Elasticsearch, the type is defined by the first value that gets indexed for a new label key. The above error indicates that the labels `name` and `hostname` are indexed as `scaled_float`, but the apm-server tries to index documents with a `string` value for these labels. If the type should be changed to `string`, then first fix the instrumentation in all your services as described above and then (assuming you are using default ILM) trigger a rollover to a new index by using the [indices-rollover api](https://www.elastic.co/guide/en/elasticsearch/reference/master/indices-rollover-index.html). The new index will create a new mapping for dynamically indexed fields, again by deriving the type from the first indexed value.
---
### Author:  [@Luis\_Garavatti](https://discuss.elastic.co/u/Luis_Garavatti)
#### Post date: [May 28, 2021, 1:49pm UTC](https://discuss.elastic.co/t/problem-with-apm-after-upgrade-to-7-10/274277/7 "2021-05-28T13:49:44Z")
Thanks for the answer Silvia.
I haven't changed anything, the only change made was the upgrade from version 7.5 to 7.10 is this type of upgrade recommended? Or would I be better off doing 7.5 to 7.6 and so on? After the upgrade it came to work for 3 days yet. Very strange
---
### Author:  [@simitt](https://discuss.elastic.co/u/simitt)
#### Post date: [May 28, 2021, 2:43pm UTC](https://discuss.elastic.co/t/problem-with-apm-after-upgrade-to-7-10/274277/8 "2021-05-28T14:43:47Z")
Upgrading from `7.5` to `7.10` should generally be fine. The errors rather indicate that something changed in the instrumented services, that the field values had been reported as `numeric` values at some point but (some) are now recorded as `string` values.
---
### Author:  [@Luis\_Garavatti](https://discuss.elastic.co/u/Luis_Garavatti)
#### Post date: [May 28, 2021, 2:56pm UTC](https://discuss.elastic.co/t/problem-with-apm-after-upgrade-to-7-10/274277/9 "2021-05-28T14:56:16Z")
All right, it's so very strange because I not changed this in my services, the fields and the metrics so created by apm-server.
How to solve this? Any idea?
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 18, 2021, 10:56am UTC](https://discuss.elastic.co/t/problem-with-apm-after-upgrade-to-7-10/274277/10 "2021-06-18T10:56:30Z")
This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.