# Move documents from one index to another

**URL:** <https://discuss.elastic.co/t/move-documents-from-one-index-to-another/201172>\
**Category:** Elasticsearch\
**Created:** [September 26, 2019, 7:46am UTC](https://discuss.elastic.co/t/move-documents-from-one-index-to-another/201172 "2019-09-26T07:46:15Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vladpov](https://avatars.discourse-cdn.com/v4/letter/v/8c91f0/32.png) [@Vladpov](https://discuss.elastic.co/u/Vladpov)\
**Post date:** [September 26, 2019, 7:46am UTC](https://discuss.elastic.co/t/move-documents-from-one-index-to-another/201172/1 "2019-09-26T07:46:15Z")

</div>

Hi guys,

I have the index where I store data. My question is whether I can restore the data that meet some conditions to a new index by using script reindexing so I can visualize the data properly.

Basically I'm thinking about something like that:

```
"script": { 
    "source": "if id has suffix = "ap" && suffix = "bn" within 60 mins move data from index1 to index2"
}

```

If I cannot do it with reindexing what can I use instead?

Thank you for any advice 🙂

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [September 26, 2019, 2:19pm UTC](https://discuss.elastic.co/t/move-documents-from-one-index-to-another/201172/2 "2019-09-26T14:19:04Z")

</div>

I'm sorry, I'm not sure I understand the requirement fully. You can use scripts, for suffix/prefix checks on the id, but that 60 minute check I am not sure about. There is no notion of a cronjob for this, but you can trigger this manually every 60 minutes, and filter documents based on a timestamp?

---

<div class="post-metadata">

**Author:** ![Vladpov](https://avatars.discourse-cdn.com/v4/letter/v/8c91f0/32.png) [@Vladpov](https://discuss.elastic.co/u/Vladpov)\
**Post date:** [September 26, 2019, 5:09pm UTC](https://discuss.elastic.co/t/move-documents-from-one-index-to-another/201172/3 "2019-09-26T17:09:35Z")

</div>

Hey @spinscale, thank you for your reply! I'm struggling with this a while 😅

In particular, we have a tons of log messages constantly coming in the following format:

```auto
Jul 23 09:24:16 mmr mmr-core[5147]: Aweg3AOMTs_1563866656876839.mt
Jul 23 09:24:18 mmr mmr-core[5210]: Aweg3AOMTs_1563866656876839.0.dn

```

There are different id numbers (1563866656876839) and two possible suffixes (mt/dn).

We parse it with logstash and store these messages in one index.

When the id number with mt suffix gots dn suffix within 1 hour it means GOOD and it should get a new field status with approved value in it. If not the field value should be disapproved.

So in the end a new index isn't needed 😃 But I'm still curious how to achieve that and if it is even possible to create and fill the new field in document based on a time condition or how to say...

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [September 30, 2019, 11:52am UTC](https://discuss.elastic.co/t/move-documents-from-one-index-to-another/201172/4 "2019-09-30T11:52:18Z")

</div>

Hey,

I see. So what I think you want is to create a connection between two log lines, like capturing start and end.

If you can parse the logline so that `Aweg3AOMTs_1563866656876839` becomes it's own field, you could maybe take a look at data frames to create a summary index, that contains information if a document has a start and end time and thus is in a good state.

See [https://www.elastic.co/guide/en/elasticsearch/reference/7.3/transforms.html](https://www.elastic.co/guide/en/elasticsearch/reference/7.3/transforms.html)

--Alex

---

<div class="post-metadata">

**Author:** ![Vladpov](https://avatars.discourse-cdn.com/v4/letter/v/8c91f0/32.png) [@Vladpov](https://discuss.elastic.co/u/Vladpov)\
**Post date:** [September 30, 2019, 3:12pm UTC](https://discuss.elastic.co/t/move-documents-from-one-index-to-another/201172/5 "2019-09-30T15:12:32Z")

</div>

Hey, thanks for help!

> I see. So what I think you want is to create a connection between two log lines, like capturing start and end.

I think that is what we need. We have loglines parsed already like that:

```
messageId: Aweg3AOMTs_1563866656871111.0.dn
text: Aweg3AOMTs
num1: 1563866656871111
num2: 0
suffix: mt/dn

```

And I have the logs in dataframes grouped by num1. Is the feature you are talking about free or part of paid X-Pack?

I'm not sure if I understand correctly the part about start and end time of a document and how is possible to give a document a state?

Thank you very much for any further clarification!

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [October 2, 2019, 8:43am UTC](https://discuss.elastic.co/t/move-documents-from-one-index-to-another/201172/6 "2019-10-02T08:43:37Z")

</div>

that is part of basic and thus free to use.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 30, 2019, 8:43am UTC](https://discuss.elastic.co/t/move-documents-from-one-index-to-another/201172/7 "2019-10-30T08:43:44Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.