# How can I save this aggregated results under a new index **URL:** \ **Category:** Elasticsearch\ **Created:** [April 12, 2024, 2:26pm UTC](https://discuss.elastic.co/t/how-can-i-save-this-aggregated-results-under-a-new-index/357299 "2024-04-12T14:26:53Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Jingyi\_Wang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jingyi_wang/32/121787_2.png) [@Jingyi\_Wang](https://discuss.elastic.co/u/Jingyi_Wang)\ **Post date:** [April 12, 2024, 2:26pm UTC](https://discuss.elastic.co/t/how-can-i-save-this-aggregated-results-under-a-new-index/357299/1 "2024-04-12T14:26:53Z")
I run the aggregation script in elastic dev tool: ```auto GET copper_scan_username_index,fibre_scan_username_index/_search { "size": 0, "aggs": { "unique_username": { "terms": { "field": "username", "min_doc_count": 2 }, "aggs": { "top_events": { "top_hits": { "size": 10 } } } } } } ``` Context of this script: it is to find out the overlap username between the two indexes by looking at if document with a username appeared on both side, then return those documents. Got the results: ```auto { "took" : 1, "timed_out" : false, "_shards" : { "total" : 2, "successful" : 2, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 4, "relation" : "eq" }, "max_score" : null, "hits" : [] }, "aggregations" : { "unique_username" : { "doc_count_error_upper_bound" : 0, "sum_other_doc_count" : 0, "buckets" : [ { "key" : "username1", "doc_count" : 2, "top_events" : { "hits" : { "total" : { "value" : 2, "relation" : "eq" }, "max_score" : 1.0, "hits" : [ { "_index" : "copper_scan_username_index", "_type" : "_doc", "_id" : "dTXSY0KNxU08jKDnZcc-nbkAAAAAAAAA", "_score" : 1.0, "_source" : { "count" : 3, "copper_scan_count" : 3, "username" : "username1" } }, { "_index" : "fibre_scan_username_index", "_type" : "_doc", "_id" : "dTXSY0KNxU08jKDnZcc-nbkAAAAAAAAA", "_score" : 1.0, "_source" : { "fibre_scan_count" : 1, "count" : 1, "username" : "username1" } } ] } } } ] } } } ``` This is what I want to save under a new index: ```auto [ { "_index" : "copper_scan_username_index", "_type" : "_doc", "_id" : "dTXSY0KNxU08jKDnZcc-nbkAAAAAAAAA", "_score" : 1.0, "_source" : { "count" : 3, "copper_scan_count" : 3, "username" : "username1" } }, { "_index" : "fibre_scan_username_index", "_type" : "_doc", "_id" : "dTXSY0KNxU08jKDnZcc-nbkAAAAAAAAA", "_score" : 1.0, "_source" : { "fibre_scan_count" : 1, "count" : 1, "username" : "username1" } } ] ``` I tried reindex, but the result saved under the new index is not what is expected. This is the reindex script: ```auto POST /_reindex { "source": { "index": ["copper_scan_username_index","fibre_scan_username_index"], "aggs": { "username_count": { "terms": { "field": "username", "min_doc_count": 2 }, "aggs": { "docs": { "top_hits": { "size": 10 } }, "bucket_filter": { "bucket_selector": { "buckets_path": { "count": "_count" }, "script": "params.count >= 2" } } } } } }, "dest": { "index": "new_index" } } ``` Can anyone help? Thanks a lot! ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [April 12, 2024, 2:53pm UTC](https://discuss.elastic.co/t/how-can-i-save-this-aggregated-results-under-a-new-index/357299/2 "2024-04-12T14:53:22Z")
Reindex does not do that. It reads the `_source` field for each document and send it to the new index. It is not meant for what you want to do. Have a look at [Create transform API | Elasticsearch Guide [8.13] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-transform.html) ---
**Author:** ![Jingyi\_Wang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jingyi_wang/32/121787_2.png) [@Jingyi\_Wang](https://discuss.elastic.co/u/Jingyi_Wang)\ **Post date:** [April 12, 2024, 4:40pm UTC](https://discuss.elastic.co/t/how-can-i-save-this-aggregated-results-under-a-new-index/357299/3 "2024-04-12T16:40:36Z")
Hi David, Thanks for pointing to Transform. I looked into how to use it. But I am struggling with how to incorporate the aggregation script into the source.query section in Transform. In a nutshell, the search result needs to be filtered based on the aggregation result. This is the aggregation query: ```auto "aggs": { "unique_username": { "terms": { "field": "username", "min_doc_count": 2 }, "aggs": { "top_events": { "top_hits": { "size": 10 } } } } } ``` The first stage aggregation can return the overlapping username between two indexes. ```auto "unique_username": { "terms": { "field": "username", "min_doc_count": 2 } } ``` The sub-aggregation is just to fetch those documents whose has one of those overlapping username. More context in terms of what documents are in the two indexes: ```auto { "_index" : "copper_scan_username_index", "_type" : "_doc", "_id" : "dTXSY0KNxU08jKDnZcc-nbkAAAAAAAAA", "_score" : 1.0, "_source" : { "count" : 3, "copper_scan_count" : 3, "username" : "username1" } }, { "_index" : "copper_scan_username_index", "_type" : "_doc", "_id" : "dUz7p-Z5C0t74EbhcYuVU-4AAAAAAAAA", "_score" : 1.0, "_source" : { "count" : 2, "copper_scan_count" : 2, "username" : "username3" } }, { "_index" : "fibre_scan_username_index", "_type" : "_doc", "_id" : "dTXSY0KNxU08jKDnZcc-nbkAAAAAAAAA", "_score" : 1.0, "_source" : { "fibre_scan_count" : 1, "count" : 1, "username" : "username1" } }, { "_index" : "fibre_scan_username_index", "_type" : "_doc", "_id" : "dRYvt34fpDoAqHahXwa4t70AAAAAAAAA", "_score" : 1.0, "_source" : { "fibre_scan_count" : 2, "count" : 2, "username" : "username2" } } ``` ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [April 12, 2024, 5:07pm UTC](https://discuss.elastic.co/t/how-can-i-save-this-aggregated-results-under-a-new-index/357299/4 "2024-04-12T17:07:59Z")
Yeah. I'm not sure you can solve this with the transform API. I never played with it myself. If you can't, then you might have to do this "manually".. ---
**Author:** ![Jingyi\_Wang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jingyi_wang/32/121787_2.png) [@Jingyi\_Wang](https://discuss.elastic.co/u/Jingyi_Wang)\ **Post date:** [April 12, 2024, 6:47pm UTC](https://discuss.elastic.co/t/how-can-i-save-this-aggregated-results-under-a-new-index/357299/5 "2024-04-12T18:47:03Z")
Let me further simplify the need. Now I only need to save this aggregated result under a new index: ```auto GET copper_scan_username_index,fibre_scan_username_index/_search { "size": 0, "aggs": { "duplicate_username": { "terms": { "field": "username", "min_doc_count": 2 } } } } ``` The aggregations.bucket data in the return is what I needs to be indexed: ```auto { "took" : 0, "timed_out" : false, "_shards" : { "total" : 2, "successful" : 2, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 4, "relation" : "eq" }, "max_score" : null, "hits" : [] }, "aggregations" : { "duplicate_username" : { "doc_count_error_upper_bound" : 0, "sum_other_doc_count" : 0, "buckets" : [ { "key" : "username1", "doc_count" : 2 } ] } } } ``` Truly appreciate if any advice can be given! ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [April 13, 2024, 6:57am UTC](https://discuss.elastic.co/t/how-can-i-save-this-aggregated-results-under-a-new-index/357299/6 "2024-04-13T06:57:19Z")
I'm not sure if that would work but look at [Transform examples | Elasticsearch Guide [8.13] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/transform-examples.html#example-customer-names). It looks similar to me. ---
**Author:** ![Jingyi\_Wang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jingyi_wang/32/121787_2.png) [@Jingyi\_Wang](https://discuss.elastic.co/u/Jingyi_Wang)\ **Post date:** [April 15, 2024, 1:36pm UTC](https://discuss.elastic.co/t/how-can-i-save-this-aggregated-results-under-a-new-index/357299/7 "2024-04-15T13:36:32Z")
Yeah, it did worked but I have to modify the aggregation logic. It gave me what I need. Thanks!