# Best way to add a new field in existing index **URL:** \ **Category:** Elasticsearch\ **Created:** [April 4, 2019, 4:33am UTC](https://discuss.elastic.co/t/best-way-to-add-a-new-field-in-existing-index/175314 "2019-04-04T04:33:29Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![msk\_76](https://avatars.discourse-cdn.com/v4/letter/m/dbc845/32.png) [@msk\_76](https://discuss.elastic.co/u/msk_76)\ **Post date:** [April 4, 2019, 4:33am UTC](https://discuss.elastic.co/t/best-way-to-add-a-new-field-in-existing-index/175314/1 "2019-04-04T04:33:29Z")
My scenario is that I have 5 fields in an existing index "emp\_data" in elasticsearch. One of these field is employee\_name. I have to add two new sixth & seventh fields "department", "company" to which this employee belongs. The dataset "emp\_data" is too big in range of 20 million documents and keep on updating in hourly basis. I have a small file containing "employee\_name","company","department" information in csv format of few kb size. I can take this file into a separate index say "corp\_data" into elasticsearch. My question is best way in elasticsearch to : 1. Join emp\_data(big datas set) with corp\_data( very small dataset) My restrictions are : I can't do a join outside ES because emp\_data is loaded by a application for which I don't have access to. I can do it only after emp\_data is already populated. Previously in a sql database I was using a ETL to join these two data sets and populate 7 fields in into a new table "emp\_record" than emp\_data and corp\_data. I am new to es. ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [April 4, 2019, 9:49am UTC](https://discuss.elastic.co/t/best-way-to-add-a-new-field-in-existing-index/175314/2 "2019-04-04T09:49:29Z")
Hey, you cannot join two indices together in elasticsearch, as Elasticsearch does not work as a SQL database. What you can do instead is to add those fields to your existing documents. One possible approach would be to use the [Update API to update parts of a document](https://www.elastic.co/guide/en/elasticsearch/reference/6.6/docs-update.html#_updates_with_a_partial_document) hope this helps! --Alex ---
**Author:** ![msk\_76](https://avatars.discourse-cdn.com/v4/letter/m/dbc845/32.png) [@msk\_76](https://discuss.elastic.co/u/msk_76)\ **Post date:** [April 5, 2019, 4:48am UTC](https://discuss.elastic.co/t/best-way-to-add-a-new-field-in-existing-index/175314/3 "2019-04-05T04:48:31Z")
I think update is useful in case I know each & every document but in my case it's a big volume of data in one index and very small volume in other. I tried to read something useful [https://www.elastic.co/guide/en/elasticsearch/reference/current/parent-join.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/parent-join.html) & [https://www.elastic.co/guide/en/elasticsearch/guide/2.x/nested-objects.html](https://www.elastic.co/guide/en/elasticsearch/guide/2.x/nested-objects.html) & Term query lookup ( joining while querying data) [https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-terms-query.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-terms-query.html) But here I am not sure which is more suitable in ES from performance point of view because I will be handing data size in magnitude of 50 million of documents during query and sorting them. Is it better to store all fields as a single document and then do query ? OR Join two documents during query with term lookup mechanism? Another point, is it possible to transform an index into another index within elasticsearch using logstash? I mean, can I query both index stored in elasticsearch ( to be joined) in logstash and join them in logstash and store in a new index that have fields from both index? ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [April 5, 2019, 8:40am UTC](https://discuss.elastic.co/t/best-way-to-add-a-new-field-in-existing-index/175314/4 "2019-04-05T08:40:57Z")
Storing and searching within single documents will always be faster (at the expense of a more complex ingestion). If I had the choice I would always try to go for the fastest solution on query time first. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 3, 2019, 8:41am UTC](https://discuss.elastic.co/t/best-way-to-add-a-new-field-in-existing-index/175314/5 "2019-05-03T08:41:00Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.