# Transform aggregation creates flattened record
**URL:** https://discuss.elastic.co/t/transform-aggregation-creates-flattened-record/309657
**Category:** Elasticsearch
**Tags:** transforms
**Created:** [July 14, 2022, 2:05pm UTC](https://discuss.elastic.co/t/transform-aggregation-creates-flattened-record/309657 "2022-07-14T14:05:02Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@dimuskin](https://discuss.elastic.co/u/dimuskin)
#### Post date: [July 14, 2022, 2:05pm UTC](https://discuss.elastic.co/t/transform-aggregation-creates-flattened-record/309657/1 "2022-07-14T14:05:02Z")
I use transformation for data preprocessing, in my case every second rps are created for each host
```auto
{
"id": "appgw-rps-host",
"source": {
"index": [
"logs-azure.platformlogs-default"
]
},
"dest": {
"index": "appgw-rps-host"
},
"frequency": "10m",
"sync": {
"time": {
"field": "@timestamp",
"delay": "5m"
}
},
"pivot": {
"group_by": {
"@timestamp": {
"date_histogram": {
"field": "@timestamp",
"calendar_interval": "1s"
}
}
},
"aggregations": {
"host": {
"terms": {
"field": "event.host"
}
}
}
}
}
```
everything works perfectly and I get the data I need. event example:
```auto
{
"_index": "appgw-rps-host",
"_id": "ADUZ-GZffUzMkUJXMRaP3wgAAAAAAAAA",
"_version": 1,
"_score": 1,
"_source": {
"@timestamp": "2022-07-12T09:57:41.000Z",
"host": {
"host1.com": 2,
"host2.com": 4,
"host3.com": 42
}
}
}
```
but this "host" type is flatten
```auto
{
"mappings": {
"_meta": {
"created_by": "transform",
},
"properties": {
"@timestamp": {
"type": "date"
},
"host": {
"type": "flattened"
}
}
}
}
```
and I can't work with them. For example, I can’t make a graph in kibana or perform any aggregation like with a numeric value. Maybe there is some possibility not to do the flattened type or do some convert using ingest pipeline?
Best Regards,
Dmitri
---
### Author:  [@dimuskin](https://discuss.elastic.co/u/dimuskin)
#### Post date: [July 15, 2022, 10:51am UTC](https://discuss.elastic.co/t/transform-aggregation-creates-flattened-record/309657/2 "2022-07-15T10:51:03Z")
looks working with index template
```auto
"dynamic_templates": [
{
"analysed_string_template": {
"path_match": "host.*",
"mapping": {
"type": "long"
}
}
}
]
```
---
### Author:  [@Hendrik\_Muhs](https://discuss.elastic.co/u/Hendrik_Muhs)
#### Post date: [July 18, 2022, 6:42am UTC](https://discuss.elastic.co/t/transform-aggregation-creates-flattened-record/309657/3 "2022-07-18T06:42:41Z")
Great you found a solution yourself.
When transform creates mappings for the destination index it does it best effort. Because `terms` is unbounded, we choose `flattened` as _default_ mapping. This data type does not run into so called "mapping explosion", meaning it it does not create too many mappings.
The default mappings aren't taken if you create the destination index yourself, directly or using an index template. The destination index is created when starting the transform for the _first_ time. This step is skipped if an existing index is found. The mappings that transform would use are part of the `_preview` API output.
---
### Author:  [@dimuskin](https://discuss.elastic.co/u/dimuskin)
#### Post date: [July 18, 2022, 8:48pm UTC](https://discuss.elastic.co/t/transform-aggregation-creates-flattened-record/309657/4 "2022-07-18T20:48:50Z")
I also noticed that first you need to make an index with at least one record (which can be deleted later) and only after that turn on the transform. otherwise, the transform ignores the mapping in the template.
---
### Author:  [@Hendrik\_Muhs](https://discuss.elastic.co/u/Hendrik_Muhs)
#### Post date: [July 19, 2022, 6:09am UTC](https://discuss.elastic.co/t/transform-aggregation-creates-flattened-record/309657/5 "2022-07-19T06:09:31Z")
Yes, the index _must_ exist, the template isn't sufficient. You should be able to create an empty index.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 16, 2022, 6:10am UTC](https://discuss.elastic.co/t/transform-aggregation-creates-flattened-record/309657/6 "2022-08-16T06:10:18Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.