# How to create an index\_2 containing Index\_1's aggregations results as fields?

**URL:** <https://discuss.elastic.co/t/how-to-create-an-index-2-containing-index-1s-aggregations-results-as-fields/227110>\
**Category:** Kibana\
**Created:** [April 8, 2020, 11:19am UTC](https://discuss.elastic.co/t/how-to-create-an-index-2-containing-index-1s-aggregations-results-as-fields/227110 "2020-04-08T11:19:45Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![oula\_oula](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/oula_oula/32/63182_2.png) [@oula\_oula](https://discuss.elastic.co/u/oula_oula)\
**Post date:** [April 8, 2020, 11:19am UTC](https://discuss.elastic.co/t/how-to-create-an-index-2-containing-index-1s-aggregations-results-as-fields/227110/1 "2020-04-08T11:19:45Z")

</div>

How to create an index\_2 containing Index\_1's aggregations results as fields ?

Is it possible to use a command like this on kibana dev tools ?

PUT /index\_2/doc\_aggs\_results\_of\_index1  
{  
field\_aggs1: ...  
field\_aggs2: ...  
{

For example:  
Index\_1 contains:  
{  
doc1:  
name : Superman  
city: New-York  
date: 01-05-2015  
}  
{  
doc2:  
name : Batman  
city: Los Angeles  
date: 22-07-2018  
}  
{  
doc3:  
name : Superman  
city: Sidney  
date: 12-10-2017  
}  
{  
doc4:  
name : Batman  
city: Paris  
date: 17-02-2018  
}

And I'd like to create index that aggregates or queries on index\_1 to make index\_2 :

Index\_2 contains:  
{  
doc1:  
name: Superman,  
cities: [  
{  
city: New-York,  
date: 01-05-2015  
},  
{  
city: Sidney,  
date: 12-10-2017  
}]  
},

{  
doc2:  
name: Batman,  
cities: [  
{  
city: Los Angeles,  
date: 22-07-2018  
},  
{  
city: Paris,  
date: 17-02-2018  
}]  
}

---

<div class="post-metadata">

**Author:** ![flash1293](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/flash1293/32/41227_2.png) [@flash1293](https://discuss.elastic.co/u/flash1293)\
**Post date:** [April 8, 2020, 12:53pm UTC](https://discuss.elastic.co/t/how-to-create-an-index-2-containing-index-1s-aggregations-results-as-fields/227110/2 "2020-04-08T12:53:27Z")

</div>

Hi, maybe dataframe transforms can help you with this: [https://www.elastic.co/guide/en/elasticsearch/reference/7.6/ecommerce-transforms.html](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/ecommerce-transforms.html)

---

<div class="post-metadata">

**Author:** ![oula\_oula](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/oula_oula/32/63182_2.png) [@oula\_oula](https://discuss.elastic.co/u/oula_oula)\
**Post date:** [April 8, 2020, 5:37pm UTC](https://discuss.elastic.co/t/how-to-create-an-index-2-containing-index-1s-aggregations-results-as-fields/227110/3 "2020-04-08T17:37:03Z")

</div>

It could have worked but unfortunatelly it seems that it only takes metric aggregations

---

<div class="post-metadata">

**Author:** ![flash1293](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/flash1293/32/41227_2.png) [@flash1293](https://discuss.elastic.co/u/flash1293)\
**Post date:** [April 8, 2020, 6:25pm UTC](https://discuss.elastic.co/t/how-to-create-an-index-2-containing-index-1s-aggregations-results-as-fields/227110/4 "2020-04-08T18:25:07Z")

</div>

Ah, I see. In this case you probably have to solve it outside of Elasticsearch- writing a script which executes the aggregation and feeds back the result into a separate index.

Maybe someone else can come up with a better solution. As this isn’t Kibana specific, it probably makes sense to repost this question in the Elasticsearch category.

---

<div class="post-metadata">

**Author:** ![Hendrik\_Muhs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendrik_muhs/32/25802_2.png) [@Hendrik\_Muhs](https://discuss.elastic.co/u/Hendrik_Muhs)\
**Post date:** [April 9, 2020, 5:32am UTC](https://discuss.elastic.co/t/how-to-create-an-index-2-containing-index-1s-aggregations-results-as-fields/227110/5 "2020-04-09T05:32:00Z")

</div>

> [@oula\_oula](#):
>
> It could have worked but unfortunatelly it seems that it only takes metric aggregations

This is not correct, transform supports other aggs as well, with `scripted_metric` - despite `metric` in the name - allows custom code which opens a lot of use cases.

We are expanding the list of supported aggs with every release. Which aggregations do you need?

---

<div class="post-metadata">

**Author:** ![oula\_oula](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/oula_oula/32/63182_2.png) [@oula\_oula](https://discuss.elastic.co/u/oula_oula)\
**Post date:** [April 9, 2020, 9:11am UTC](https://discuss.elastic.co/t/how-to-create-an-index-2-containing-index-1s-aggregations-results-as-fields/227110/6 "2020-04-09T09:11:17Z")

</div>

I need terms aggregation but the dev tools manager answers to me:

"error": {  
"root\_cause": [  
{  
"type": "status\_exception",  
"reason": "Unsupported aggregation type [terms]"  
}

---

<div class="post-metadata">

**Author:** ![Hendrik\_Muhs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendrik_muhs/32/25802_2.png) [@Hendrik\_Muhs](https://discuss.elastic.co/u/Hendrik_Muhs)\
**Post date:** [April 9, 2020, 9:32am UTC](https://discuss.elastic.co/t/how-to-create-an-index-2-containing-index-1s-aggregations-results-as-fields/227110/7 "2020-04-09T09:32:48Z")

</div>

thanks, we already have an [enhancement request](https://github.com/elastic/elasticsearch/issues/51073) for terms.

In case its a small and known set of terms you can use a `filter` aggregation in the upcoming `7.7`, see this [example](https://www.elastic.co/guide/en/elasticsearch/reference/7.7/transform-examples.html#example-clientips).

As already pointed out, with `scripted_metric` you could create a `HashSet` to map/reduce a terms frequency list.

I can not say when `terms` support in pivot will be available, but as this is a top ask: hopefully soon.

---

<div class="post-metadata">

**Author:** ![oula\_oula](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/oula_oula/32/63182_2.png) [@oula\_oula](https://discuss.elastic.co/u/oula_oula)\
**Post date:** [April 9, 2020, 9:51am UTC](https://discuss.elastic.co/t/how-to-create-an-index-2-containing-index-1s-aggregations-results-as-fields/227110/8 "2020-04-09T09:51:12Z")

</div>

Filter doesn't adress my issue because it works by mentionning a specific field value, but I need to gather docs by field name.

---

<div class="post-metadata">

**Author:** ![Hendrik\_Muhs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendrik_muhs/32/25802_2.png) [@Hendrik\_Muhs](https://discuss.elastic.co/u/Hendrik_Muhs)\
**Post date:** [April 9, 2020, 11:52am UTC](https://discuss.elastic.co/t/how-to-create-an-index-2-containing-index-1s-aggregations-results-as-fields/227110/9 "2020-04-09T11:52:14Z")

</div>

For bucketing you use `group_by`, which supports grouping by term. For the collapsing part you can use a `scripted_metric`.

Here is an example configuration. The only thing missing according to your start post is dropping the field `name`.

```auto
POST _transform/_preview
{
  "source": {
    "index": "index-1"
  },
  "dest" : { 
    "index" : "index-2"
  },
  "pivot": {
    "group_by": {  
      "clientip": { "terms": { "field": "name" } }
      },
  "aggregations": {
    "all_docs": {
      "scripted_metric": {
        "init_script": "state.docs = []",
        "map_script": "state.docs.add(new HashMap(params['_source']))",
        "combine_script": "return state.docs",
        "reduce_script": "def docs = []; for (s in states) {for (d in s) { docs.add(d);}}return docs"
      }
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 7, 2020, 12:00pm UTC](https://discuss.elastic.co/t/how-to-create-an-index-2-containing-index-1s-aggregations-results-as-fields/227110/10 "2020-05-07T12:00:44Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.