# Creating an index from another index with only certain fields and a different ID than the original index

**URL:** <https://discuss.elastic.co/t/creating-an-index-from-another-index-with-only-certain-fields-and-a-different-id-than-the-original-index/247576>\
**Category:** Elasticsearch\
**Created:** [September 4, 2020, 4:15pm UTC](https://discuss.elastic.co/t/creating-an-index-from-another-index-with-only-certain-fields-and-a-different-id-than-the-original-index/247576 "2020-09-04T16:15:03Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![edster](https://avatars.discourse-cdn.com/v4/letter/e/da6949/32.png) [@edster](https://discuss.elastic.co/u/edster)\
**Post date:** [September 4, 2020, 4:15pm UTC](https://discuss.elastic.co/t/creating-an-index-from-another-index-with-only-certain-fields-and-a-different-id-than-the-original-index/247576/1 "2020-09-04T16:15:03Z")

</div>

Is there anyway to create an index from another index but only taking certain fields, not all? The new index would have a new ID as well and include a new alias. I was looking at the reindex API and the clone API as maybe a possible solution however not sure if this is able to accomplish this. Another possible solution could be somehow making text file from the index and then transforming that file as needed to then reingest into the new index, however, this process seems to be the longest in terms of time as the original index may have more than 20 million records/documents.

An example of a document in the original document would be:

\_id: field1-field2-field3-field4  
field1: \*\*\*  
field2: \*\*\*  
field3: \*\*\*\*\*  
field4: \*\*\*\*  
field5: \*\*\*\*\*  
field6: \*\*\*\*\*\*  
field7: \*\*\*\*\*\*  
field8: \*\*\*\*\*\*

The new index would then have :

\_id: field2  
field1: \*\*\*  
field2: \*\*\*  
field6: \*\*\*\*\*\*  
field8: \*\*\*\*\*\*

The example is only for one record but it would be for the whole index, all documents in that index.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 6, 2020, 4:53am UTC](https://discuss.elastic.co/t/creating-an-index-from-another-index-with-only-certain-fields-and-a-different-id-than-the-original-index/247576/3 "2020-09-06T04:53:53Z")

</div>

You should be able to use the reindex API together with an ingest pipeline that removed the unwanted fields.

---

<div class="post-metadata">

**Author:** ![edster](https://avatars.discourse-cdn.com/v4/letter/e/da6949/32.png) [@edster](https://discuss.elastic.co/u/edster)\
**Post date:** [September 8, 2020, 12:08pm UTC](https://discuss.elastic.co/t/creating-an-index-from-another-index-with-only-certain-fields-and-a-different-id-than-the-original-index/247576/4 "2020-09-08T12:08:35Z")

</div>

When you say an ingest pipeline is that like referring to making another logstash config file with essentially remove fields options in it and then running that ingest process after the reindex API REST calls have been made?

---

<div class="post-metadata">

**Author:** ![edster](https://avatars.discourse-cdn.com/v4/letter/e/da6949/32.png) [@edster](https://discuss.elastic.co/u/edster)\
**Post date:** [September 18, 2020, 4:02pm UTC](https://discuss.elastic.co/t/creating-an-index-from-another-index-with-only-certain-fields-and-a-different-id-than-the-original-index/247576/5 "2020-09-18T16:02:25Z")

</div>

The reindex is working however when trying to use this in the script portion to get the date from the previous index name to add to the new one it doesnt add it.

> "ctx.\_index = ctx.\_index + '-' + (ctx.\_index.substring('testsubindex-'.length(), ctx.\_index.length()))"

I got this from the docs however this is giving me an error

> "error" : {  
> "root\_cause" : [  
> {  
> "type" : "script\_exception",  
> "reason" : "runtime error",  
> "script\_stack" : [  
> "ctx.\_index = ctx.\_index + - + (ctx.\_index.substring('testsubindex-'.length(), ctx.\_index.length()))",  
> " ^---- HERE"  
> ],  
> "script" : "ctx.\_index = ctx.\_index + - + (ctx.\_index.substring('testsubindex-'.length(), ctx.\_index.length()))",  
> "lang" : "painless"  
> }  
> ],  
> "type" : "script\_exception",  
> "reason" : "runtime error",  
> "script\_stack" : [  
> "ctx.\_index = ctx.\_index + - + (ctx.\_index.substring('testsubindex-'.length(), ctx.\_index.length()))",  
> " ^---- HERE"  
> ],  
> "script" : "ctx.\_index = ctx.\_index + - + (ctx.\_index.substring('testsubindex-'.length(), ctx.\_index.length()))",  
> "lang" : "painless",  
> "caused\_by" : {  
> "type" : "null\_pointer\_exception",  
> "reason" : null  
> }  
> },  
> "status" : 400

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 16, 2020, 4:02pm UTC](https://discuss.elastic.co/t/creating-an-index-from-another-index-with-only-certain-fields-and-a-different-id-than-the-original-index/247576/6 "2020-10-16T16:02:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.