# Enrich documents by copying fields from another index
**URL:** https://discuss.elastic.co/t/enrich-documents-by-copying-fields-from-another-index/170882
**Category:** Elasticsearch
**Created:** [March 5, 2019, 11:39am UTC](https://discuss.elastic.co/t/enrich-documents-by-copying-fields-from-another-index/170882 "2019-03-05T11:39:53Z")
**Posts on this page:** 12
**Page:** 1
### Author:  [@biconou](https://discuss.elastic.co/u/biconou)
#### Post date: [March 5, 2019, 11:39am UTC](https://discuss.elastic.co/t/enrich-documents-by-copying-fields-from-another-index/170882/1 "2019-03-05T11:39:53Z")
Hi,
I got 2 indices.
say index1 and index2.
When I index a new document in index2, I want some fields to be copied from the document in index1 having the same \_id.
I previously realized this using logstash + the elasticsearch filter.
But this time, i'm not using logstash. I index documents in index2 using the bulk api.
What is the best way to achieve that ? Ingestion plugin ?
Thanks.
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [March 5, 2019, 5:17pm UTC](https://discuss.elastic.co/t/enrich-documents-by-copying-fields-from-another-index/170882/2 "2019-03-05T17:17:41Z")
> What is the best way to achieve that ?
I think that what you did in the past is a good way to solve this problem. I mean:
> I previously realized this using logstash + the elasticsearch filter.
That's probably what I'd do.
---
### Author:  [@biconou](https://discuss.elastic.co/u/biconou)
#### Post date: [March 6, 2019, 10:24am UTC](https://discuss.elastic.co/t/enrich-documents-by-copying-fields-from-another-index/170882/3 "2019-03-06T10:24:00Z")
Thanks @dadoonet but I want to get rid of logstash exactly as you describe here :
I would like to include in the ingest node pipeline some plugin that is able to enrich datas from existing documents in the cluster.
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [March 6, 2019, 11:12am UTC](https://discuss.elastic.co/t/enrich-documents-by-copying-fields-from-another-index/170882/4 "2019-03-06T11:12:38Z")
Yeah. I'm not doing that talk anymore. That ended up being a bad idea after all to use ingest to do lookups in a 3rd party system. I was more using that for demo purpose, to introduce what ingest is and how you can easily build your own plugin but the use case was not the best one. ☹
You can see what is my recommendation today (using Logstash):
- [http://david.pilato.fr/blog/2018/03/22/enriching-your-postal-addresses-with-elastic-stack-part-1/](http://david.pilato.fr/blog/2018/03/22/enriching-your-postal-addresses-with-elastic-stack-part-1/)
- [http://david.pilato.fr/blog/2018/03/24/enriching-your-postal-addresses-with-elastic-stack-part-2/](http://david.pilato.fr/blog/2018/03/24/enriching-your-postal-addresses-with-elastic-stack-part-2/)
- [http://david.pilato.fr/blog/2018/03/24/enriching-your-postal-addresses-with-elastic-stack-part-3/](http://david.pilato.fr/blog/2018/03/24/enriching-your-postal-addresses-with-elastic-stack-part-3/)
If you really want to go that way (which I do not recommend), you can look at this PR which was a WIP but was never merged because of the reasons I exposed before.
>
---
### Author:  [@biconou](https://discuss.elastic.co/u/biconou)
#### Post date: [March 6, 2019, 11:27am UTC](https://discuss.elastic.co/t/enrich-documents-by-copying-fields-from-another-index/170882/5 "2019-03-06T11:27:10Z")
@dadoonet Thanks for the share.
I will also reconsider using logstash.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 3, 2019, 11:40am UTC](https://discuss.elastic.co/t/enrich-documents-by-copying-fields-from-another-index/170882/6 "2019-04-03T11:40:04Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [November 26, 2019, 9:15am UTC](https://discuss.elastic.co/t/enrich-documents-by-copying-fields-from-another-index/170882/7 "2019-11-26T09:15:42Z")
---
### Author:  [@val](https://discuss.elastic.co/u/val)
#### Post date: [November 26, 2019, 9:19am UTC](https://discuss.elastic.co/t/enrich-documents-by-copying-fields-from-another-index/170882/8 "2019-11-26T09:19:51Z")
Guys,
Actually, the feature you're asking for is coming in 7.5 via the new [enrich processor](https://www.elastic.co/guide/en/elasticsearch/reference/7.5/ingest-enriching-data.html), which kind of provides index-time JOIN capability.
The main idea is to set up an [enrich policy](https://www.elastic.co/guide/en/elasticsearch/reference/7.5/enrich-setup.html#create-enrich-policy) that will source data from your related indexes into a new "enrich index" and then you can leverage that "enrich index" in your ingest pipeline using an [enrich processor](https://www.elastic.co/guide/en/elasticsearch/reference/7.5/enrich-processor.html) in order to enrich your documents with related fields.
So, without going in too many details, here is how it works in practice:
1. You have an index `A` with fields (`a`, `b`, `c`, `d`) that you'd like to use for enriching your incoming documents
2. You define an enrich policy based on that index `A` and the "join" field `a`
3. You define an ingest pipeline with an enrich processor that will try to match field `z` of the incoming document against field `A.a` of the enrich index
4. If a match is found, your incoming document will get fields `b`, `c` and `d` from the index `A`. Note that it will also get the match field `a` that you can remove using a `remove` processor if needed.
That should pretty much work the way you expect. You can find a complete example [here](https://www.elastic.co/guide/en/elasticsearch/reference/7.5/match-enrich-policy-type.html). At the beginning, it will work for exact matches (i.e. `term` query) and geo matches (i.e. `geo_shape` query), but they will probably add new kind of matches (like [`range` matches](https://github.com/elastic/elasticsearch/issues/48988)) in the near future.
---
### Author:  [@val](https://discuss.elastic.co/u/val)
#### Post date: [December 2, 2019, 8:54pm UTC](https://discuss.elastic.co/t/enrich-documents-by-copying-fields-from-another-index/170882/9 "2019-12-02T20:54:09Z")
Elastic Stack 7.5 has been [released today](https://www.elastic.co/blog/elastic-stack-7-5-0-released) and enrich processors are now available.
---
### Author:  [@crickes](https://discuss.elastic.co/u/crickes)
#### Post date: [February 5, 2020, 5:11pm UTC](https://discuss.elastic.co/t/enrich-documents-by-copying-fields-from-another-index/170882/10 "2020-02-05T17:11:36Z")
Will it be possible to use an analysed match rather then a term query to join the databases at all? I have a use case where I want to enrich a document with the best match (based on the '\_score' value), as opposed to using a term query.
---
### Author:  [@val](https://discuss.elastic.co/u/val)
#### Post date: [February 12, 2020, 7:59am UTC](https://discuss.elastic.co/t/enrich-documents-by-copying-fields-from-another-index/170882/11 "2020-02-12T07:59:44Z")
I don't think it's planned at the moment. The matching currently only occurs on yes/no filters and not based on scoring. But I see you've created a [feature request](https://github.com/elastic/elasticsearch/issues/51978) so we'll see where that leads us 😉
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 4, 2022, 6:57am UTC](https://discuss.elastic.co/t/enrich-documents-by-copying-fields-from-another-index/170882/12 "2022-11-04T06:57:45Z")