# Copy speciifc data to another index automatically

**URL:** https://discuss.elastic.co/t/copy-speciifc-data-to-another-index-automatically/314300
**Category:** Elasticsearch
**Created:** [September 13, 2022, 1:53pm UTC](https://discuss.elastic.co/t/copy-speciifc-data-to-another-index-automatically/314300 "2022-09-13T13:53:17Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![queried1](https://avatars.discourse-cdn.com/v4/letter/q/278dde/32.png) [@queried1](https://discuss.elastic.co/u/queried1)
#### Post date: [September 13, 2022, 1:53pm UTC](https://discuss.elastic.co/t/copy-speciifc-data-to-another-index-automatically/314300/1 "2022-09-13T13:53:17Z")

</div>

Hello!

Currently I use FIlebeat to send data to Logstash and then to Elasticsearch. There are mostly application logs that contain a lot of data. Everything is sent to an index called filebeat--0000x and is rolled over and kept for 3 months.

Sometimes there are very important logs that need to saved for much longer and it would be better to save them in a separate index that has a completely different ILM policy.

My first idea is to write a script that would query specific results, check if a "\_docid" exists in the new index and if not, then copy the document over and remove unneeded fields. This would create an index that has only the needed data and would not take up much disk space.

Maybe there is a much more elegant way to "copy" only needed data to a separate index?

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [September 13, 2022, 2:00pm UTC](https://discuss.elastic.co/t/copy-speciifc-data-to-another-index-automatically/314300/2 "2022-09-13T14:00:15Z")

</div>

Maybe a Transform can help you do that.

Check the [documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/transforms.html) about transforms.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 11, 2022, 2:01pm UTC](https://discuss.elastic.co/t/copy-speciifc-data-to-another-index-automatically/314300/3 "2022-10-11T14:01:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
