# Cannot start transform job - Cannot set after key in the composite aggregation \[\_transform\]
**URL:** https://discuss.elastic.co/t/cannot-start-transform-job-cannot-set-after-key-in-the-composite-aggregation-transform/283013
**Category:** Elasticsearch
**Tags:** transforms
**Created:** [September 1, 2021, 8:37am UTC](https://discuss.elastic.co/t/cannot-start-transform-job-cannot-set-after-key-in-the-composite-aggregation-transform/283013 "2021-09-01T08:37:14Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@ciment](https://discuss.elastic.co/u/ciment)
#### Post date: [September 1, 2021, 8:37am UTC](https://discuss.elastic.co/t/cannot-start-transform-job-cannot-set-after-key-in-the-composite-aggregation-transform/283013/1 "2021-09-01T08:37:14Z")
Hi,
We would like to setup transform job to create statistics - how many devices are connected per hour. In our code we have added for every request, deviceId and clientId to labels data [labels.deviceId, labels.clientId].
We have created index template:
```auto
PUT _index_template/test_apm_devices_visits_per_hour
{
"template": {
"settings": {
"number_of_shards": 10,
"number_of_replicas": 1
},
"mappings": {
"properties": {
"labels.deviceId": {
"type": "keyword"
},
"labels.clientId": {
"type": "keyword"
},
"@timestamp": {
"type": "date"
},
"visits": {
"type": "long"
}
}
}
},
"index_patterns": [
"test_apm_devices_visits_per_hour-*"
]
}
```
We have created the tansform:
```auto
PUT _transform/test_devices-visits-per-hour
{
"source": {
"index": "apm-*",
"query": {
"bool": {
"must": [
{
"range": {
"@timestamp": {
"gte": "now-1d/d"
}
}
},
{
"wildcard": {
"service.name": {
"value": "prefix-for-service-name*"
}
}
}
]
}
}
},
"dest": {
"index": "test_apm_devices_visits_per_hour"
},
"pivot": {
"group_by": {
"labels.deviceId": {
"terms": {
"field": "labels.deviceId"
}
},
"labels.clientId": {
"terms": {
"field": "labels.clientId"
}
},
"@timestamp": {
"date_histogram": {
"field": "@timestamp",
"calendar_interval": "1h"
}
},
"service.name": {
"terms": {
"field": "service.name"
}
}
},
"aggregations": {
"visits": {
"value_count": {
"field": "labels.deviceId"
}
}
}
},
"frequency": "30m",
"sync": {
"time": {
"field": "@timestamp",
"delay": "1m"
}
},
"settings": {
"max_page_search_size": 10000
}
}
```
Creating index from template:
```auto
PUT test_apm_devices_visits_per_hour
```
Run transformation:
```auto
POST _transform/test_devices-visits-per-hour/_start
```
Immediately after start, we received exception.
```auto
task encountered irrecoverable failure: ElasticsearchParseException[Cannot set after key in the composite aggregation [_transform] - incompatible value in the position 0: invalid value, expected string, got Double]; nested: IllegalArgumentException[incompatible value in the position 0: invalid value, expected string, got Double]; nested: IllegalArgumentException[invalid value, expected string, got Double];; java.lang.IllegalArgumentException: incompatible value in the position 0: invalid value, expected string, got Double
```
Any help? Thanks in advance
---
### Author:  [@sophie\_chang](https://discuss.elastic.co/u/sophie_chang)
#### Post date: [September 2, 2021, 8:41am UTC](https://discuss.elastic.co/t/cannot-start-transform-job-cannot-set-after-key-in-the-composite-aggregation-transform/283013/2 "2021-09-02T08:41:18Z")
When a pivot transform starts, it will create a destination index based on deduced mappings. It will use mappings deduced from its config and the source destination fields. e.g. sum(half\_float) will be mapped to a float. It does not use the template or dynamic mappings (except for scripted fields).
By using transform `_preview` you can see the expected `generated_dest_index` mappings. I would recommend trying this as the next troubleshooting step.
> **[Preview transform API | Elasticsearch Guide \[7.14\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/preview-transform.html)**
If you wish to avoid mapping deduction, then create the empty destination index before starting the transform.
Hope this helps.
---
### Author:  [@ciment](https://discuss.elastic.co/u/ciment)
#### Post date: [September 2, 2021, 2:20pm UTC](https://discuss.elastic.co/t/cannot-start-transform-job-cannot-set-after-key-in-the-composite-aggregation-transform/283013/3 "2021-09-02T14:20:11Z")
Hi Sophie,
I have tried preview of transform. This is part of the response:
```auto
...
{
"visits" : 63,
"@timestamp" : "2021-09-02T09:00:00.000Z",
"service" : {
"name" : "my-service-name"
},
"labels" : {
"clientId" : 3.9746313E7,
"deviceId" : 492060.0
}
}
],
"generated_dest_index" : {
"mappings" : {
"_meta" : {
"_transform" : {
"transform" : "transform-preview",
"version" : {
"created" : "7.14.0"
},
"creation_date_in_millis" : 1630592219862
},
"created_by" : "transform"
},
"properties" : {
"visits" : {
"type" : "long"
},
"@timestamp" : {
"type" : "date"
},
"service.name" : {
"type" : "keyword"
},
"labels.clientId" : {
"type" : "scaled_float"
},
"service" : {
"type" : "object"
},
"labels.deviceId" : {
"type" : "scaled_float"
},
"labels" : {
"type" : "object"
}
}
},
"settings" : {
"index" : {
"number_of_shards" : "1",
"auto_expand_replicas" : "0-1"
}
},
"aliases" : { }
}
```
I have changed the index template mapping properties to:
```auto
PUT _index_template/apm_devices_visits_per_hour
{
"template": {
"settings": {
"number_of_shards": 10,
"number_of_replicas": 1
},
"mappings": {
"properties": {
"visits" : {
"type" : "long"
},
"@timestamp" : {
"type" : "date"
},
"service.name" : {
"type" : "keyword"
},
"labels.clientId" : {
"type" : "scaled_float",
"scaling_factor": 10000000
},
"service" : {
"type" : "object"
},
"labels.deviceId" : {
"type" : "scaled_float",
"scaling_factor": 10000000
},
"labels" : {
"type" : "object"
}
}
}
},
"index_patterns": [
"apm_devices_visits_per_hour*"
]
}
```
And also I created destination index.
```auto
PUT apm_devices_visits_per_hour
```
After starting the transform I got the same exception.
```auto
task encountered irrecoverable failure: ElasticsearchParseException[Cannot set after key in the composite aggregation [_transform] - incompatible value in the position 0: invalid value, expected string, got Double]; nested: IllegalArgumentException[incompatible value in the position 0: invalid value, expected string, got Double]; nested: IllegalArgumentException[invalid value, expected string, got Double];; java.lang.IllegalArgumentException: incompatible value in the position 0: invalid value, expected string, got Double
```
Any help? Thanks.
---
### Author:  [@BenTrent](https://discuss.elastic.co/u/BenTrent)
#### Post date: [September 3, 2021, 12:15pm UTC](https://discuss.elastic.co/t/cannot-start-transform-job-cannot-set-after-key-in-the-composite-aggregation-transform/283013/4 "2021-09-03T12:15:13Z")
Hey @ciment,
Your exception might have to do with your source index mappings.
I am thinking that your `labels.deviceId` is mapped as a number in one `apm-*` index and as a `keyword` in another
Can you execute
```auto
GET apm-*/_field_caps?field=labels.deviceId,labels.clientId,@timestamp,service.name
```
And verify that the field types are unified across your indices?
Thanks!
---
### Author:  [@ciment](https://discuss.elastic.co/u/ciment)
#### Post date: [September 6, 2021, 6:02am UTC](https://discuss.elastic.co/t/cannot-start-transform-job-cannot-set-after-key-in-the-composite-aggregation-transform/283013/5 "2021-09-06T06:02:40Z")
Hi @BenTrent,
I verified the field types across my indices. I found that fields (labels.deviceId,labels.clientId) exist only for apm transaction and apm error indices.
I edited my transform and in settings I changed from
```auto
...
"index": "apm-*",
...
```
to
```auto
...
"index": ["apm-*-transaction*","apm-*-error*"],
...
```
and finally my transform **works.**
Thanks @BenTrent and @sophie_chang for your advices.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 4, 2021, 6:03am UTC](https://discuss.elastic.co/t/cannot-start-transform-job-cannot-set-after-key-in-the-composite-aggregation-transform/283013/6 "2021-10-04T06:03:20Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.