# Aggregate two records in one index **URL:** \ **Category:** Elasticsearch\ **Tags:** elastic-stack-monitoring\ **Created:** [October 21, 2023, 12:25pm UTC](https://discuss.elastic.co/t/aggregate-two-records-in-one-index/345503 "2023-10-21T12:25:41Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![reza\_sabz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/reza_sabz/32/115443_2.png) [@reza\_sabz](https://discuss.elastic.co/u/reza_sabz)\ **Post date:** [October 21, 2023, 12:25pm UTC](https://discuss.elastic.co/t/aggregate-two-records-in-one-index/345503/1 "2023-10-21T12:25:41Z")
Hello guys, I have an index with a lot of records, like these: "\_source": { "terminal\_number": " 123456", "date": "2023-10-18 12:02:31.676", "iin": " 111111111 ", "service\_type": "o.t.s.transactions.trm.TerminalService", "log\_level": "INFO", "microservice": "microservices-transaction", "iin\_data": "transaction-service,7ec6ea6dyc1e3f11,7ec6eq6ddc1e3f11", "@timestamp": "2023-10-18T08:10:34.267Z" } and other record: "\_source": { "date": "2023-10-18 12:14:31.163", "role": " ROLE\_PSP", "service\_type": "o.t.s.t.security.TokenService", "log\_level": "INFO", "microservice": "microservices-transaction-svc", "iin\_data": "transaction-service,7ec6ea6dyc1e3f11,7ec6eq6ddc1e3f11", "@timestamp": "2023-10-18T08:987:34.266Z" } iin\_data in some of records maybe same, now I want to find same of iin\_data and aggregate them in an index Can anyone help? ---
**Author:** ![carly.richmond](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carly.richmond/32/104935_2.png) [@carly.richmond](https://discuss.elastic.co/u/carly.richmond)\ **Post date:** [October 23, 2023, 8:51am UTC](https://discuss.elastic.co/t/aggregate-two-records-in-one-index/345503/2 "2023-10-23T08:51:57Z")
Hi @reza_sabz, Welcome back! Have you had a look at [transforms](https://www.elastic.co/guide/en/elasticsearch/reference/current/transforms.html) to see if this can give you the summarized index that you need? It's not a feature I've played with much but might be worth a look. Let us know how you get on! ---
**Author:** ![reza\_sabz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/reza_sabz/32/115443_2.png) [@reza\_sabz](https://discuss.elastic.co/u/reza_sabz)\ **Post date:** [October 23, 2023, 9:29am UTC](https://discuss.elastic.co/t/aggregate-two-records-in-one-index/345503/3 "2023-10-23T09:29:33Z")
Thank you for reply 😇 I tried to solve my issue with enrich policy, for example: I have two index: > **index\_1 :** ```auto { "took": 1, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 3, "relation": "eq" }, "max_score": 1, "hits": [ { "_index": "index-1-2023.10.22", "_id": "BRxNV4sB4G6JLjkwUiLj", "_score": 1, "_source": { "terminal_number": "12245657", "iin": " 111111123 ", "microservice": "microservices-transaction-svc", "date": "2023-10-22 15:42:45.136", "@timestamp": "2023-10-22T12:12:45.417Z", "iin_data": "a1552135cd3d16db", "log_level": "INFO", "service_type": "o.t.s.transactions.trm.TerminalService" } }, { "_index": "index-1-2023.10.22", "_id": "U_NNV4sB-5gOz7rQUrHj", "_score": 1, "_source": { "terminal_number": "12245657", "iin": " 111111123 ", "microservice": "microservices-transaction-svc", "date": "2023-10-22 15:42:44.646", "@timestamp": "2023-10-22T12:12:45.417Z", "iin_data": "8fc72d10c3952962", "log_level": "INFO", "service_type": "o.t.s.transactions.trm.TerminalService" } }, { "_index": "index-1-2023.10.22", "_id": "BxxNV4sB4G6JLjkwWCKQ", "_score": 1, "_source": { "terminal_number": "12245657", "iin": " 111111123 ", "microservice": "microservices-transaction-svc", "date": "2023-10-22 15:42:45.672", "@timestamp": "2023-10-22T12:12:47.414Z", "iin_data": "1d1f7bb4e4fd7f28", "log_level": "INFO", "service_type": "o.t.s.transactions.trm.TerminalService" } } ] } } ``` > **index-2 :** ```auto { "took": 2, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 1, "relation": "eq" }, "max_score": 1, "hits": [ { "_index": "index-2-2023.10.22", "_id": "UvNNV4sB-5gOz7rQUrGp", "_score": 1, "_source": { "microservice": "microservices-transaction-svc", "date": "2023-10-22 15:42:44.634", "@timestamp": "2023-10-22T12:12:45.416Z", "uri_query": "/itemorder", "iin_data": "8fc72d10c3952962", "log_level": "INFO" } } ] } } ``` > **my enrich policy:** ```auto PUT _enrich/policy/merge { "match": { "indices": "index-2-2023.10.22", "match_field": "iin_data", "enrich_fields": ["uri_query"] } } ``` ```auto POST /_enrich/policy/merge/_execute ``` ```auto PUT _ingest/pipeline/enrich { "processors": [ { "enrich": { "description": "Add 'uri_query' data based on 'iin_data'", "policy_name": "merge", "field": "iin_data", "target_field": "new", "max_matches": "1" } } ] } ``` ```auto POST _reindex { "source": { "index": "index-1-2023.10.22" }, "dest": { "index": "merge", "pipeline": "enrich" } } ``` But it doesn't work. ---
**Author:** ![carly.richmond](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carly.richmond/32/104935_2.png) [@carly.richmond](https://discuss.elastic.co/u/carly.richmond)\ **Post date:** [October 23, 2023, 9:39am UTC](https://discuss.elastic.co/t/aggregate-two-records-in-one-index/345503/4 "2023-10-23T09:39:39Z")
> [@reza\_sabz](#): > > But it doesn't work. I would be interested to hear what you mean by it doesn't work. But enrich policies are intended to enrich documents by adding fields from another index based on a particular field match rather than against the same index, so I would expect some interesting results when trying to enrich against the same index. ---
**Author:** ![reza\_sabz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/reza_sabz/32/115443_2.png) [@reza\_sabz](https://discuss.elastic.co/u/reza_sabz)\ **Post date:** [October 23, 2023, 9:47am UTC](https://discuss.elastic.co/t/aggregate-two-records-in-one-index/345503/5 "2023-10-23T09:47:20Z")
At first I tried to collect some of their records as well, but I didn't get good results. Because of this, I made 2 indexes and tried to collect them ---
**Author:** ![carly.richmond](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carly.richmond/32/104935_2.png) [@carly.richmond](https://discuss.elastic.co/u/carly.richmond)\ **Post date:** [October 23, 2023, 9:49am UTC](https://discuss.elastic.co/t/aggregate-two-records-in-one-index/345503/6 "2023-10-23T09:49:01Z")
Ok, did splitting into two indices work for you in the end? ---
**Author:** ![reza\_sabz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/reza_sabz/32/115443_2.png) [@reza\_sabz](https://discuss.elastic.co/u/reza_sabz)\ **Post date:** [October 23, 2023, 9:52am UTC](https://discuss.elastic.co/t/aggregate-two-records-in-one-index/345503/7 "2023-10-23T09:52:42Z")
No, this is the result: ```auto { "took": 2, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 3, "relation": "eq" }, "max_score": 1, "hits": [ { "_index": "merge", "_id": "BRxNV4sB4G6JLjkwUiLj", "_score": 1, "_source": { "date": "2023-10-22 15:42:45.136", "service_type": "o.t.s.transactions.trm.TerminalService", "@timestamp": "2023-10-22T12:12:45.417Z", "iin_data": "a1552135cd3d16db", "terminal_number": "12245657", "microservice": "microservices-transaction-svc", "log_level": "INFO", "iin": " 111111123 " } }, { "_index": "merge", "_id": "U_NNV4sB-5gOz7rQUrHj", "_score": 1, "_source": { "date": "2023-10-22 15:42:44.646", "service_type": "o.t.s.transactions.trm.TerminalService", "@timestamp": "2023-10-22T12:12:45.417Z", "iin_data": "8fc72d10c3952962", "terminal_number": "12245657", "microservice": "microservices-transaction-svc", "log_level": "INFO", "iin": " 111111123 " } }, { "_index": "merge", "_id": "BxxNV4sB4G6JLjkwWCKQ", "_score": 1, "_source": { "date": "2023-10-22 15:42:45.672", "service_type": "o.t.s.transactions.trm.TerminalService", "@timestamp": "2023-10-22T12:12:47.414Z", "iin_data": "1d1f7bb4e4fd7f28", "terminal_number": "12245657", "microservice": "microservices-transaction-svc", "log_level": "INFO", "iin": " 111111123 " } } ] } } ``` It returned the same records to me! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 20, 2023, 9:52am UTC](https://discuss.elastic.co/t/aggregate-two-records-in-one-index/345503/8 "2023-11-20T09:52:42Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.