# Extract a subset of an big Index

**URL:** <https://discuss.elastic.co/t/extract-a-subset-of-an-big-index/236360>\
**Category:** Logstash\
**Created:** [June 9, 2020, 2:21pm UTC](https://discuss.elastic.co/t/extract-a-subset-of-an-big-index/236360 "2020-06-09T14:21:47Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![ginu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ginu/32/86297_2.png) [@ginu](https://discuss.elastic.co/u/ginu)\
**Post date:** [June 9, 2020, 2:21pm UTC](https://discuss.elastic.co/t/extract-a-subset-of-an-big-index/236360/1 "2020-06-09T14:21:47Z")

</div>

Hi  
I have a bit index journalbeat-\*... we would like to have an extract of it which we would like to keep for months with only a few fields... what i the best way to achieve this?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 9, 2020, 3:48pm UTC](https://discuss.elastic.co/t/extract-a-subset-of-an-big-index/236360/2 "2020-06-09T15:48:17Z")

</div>

You could reindex the data you want into a new index. Use an elasticsearch input, and perhaps a prune filter to whitelist the few fields that you want to keep.

---

<div class="post-metadata">

**Author:** ![ginu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ginu/32/86297_2.png) [@ginu](https://discuss.elastic.co/u/ginu)\
**Post date:** [June 10, 2020, 4:38am UTC](https://discuss.elastic.co/t/extract-a-subset-of-an-big-index/236360/3 "2020-06-10T04:38:23Z")

</div>

Thank you Badger... but can this be done continuously every hour or so?

---

<div class="post-metadata">

**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\
**Post date:** [June 10, 2020, 6:53am UTC](https://discuss.elastic.co/t/extract-a-subset-of-an-big-index/236360/4 "2020-06-10T06:53:20Z")

</div>

You just have to configure the right schedule: [https://www.elastic.co/guide/en/logstash/current/plugins-inputs-elasticsearch.html#\_scheduling](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-elasticsearch.html#_scheduling)

---

<div class="post-metadata">

**Author:** ![ginu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ginu/32/86297_2.png) [@ginu](https://discuss.elastic.co/u/ginu)\
**Post date:** [June 10, 2020, 8:24am UTC](https://discuss.elastic.co/t/extract-a-subset-of-an-big-index/236360/5 "2020-06-10T08:24:51Z")

</div>

Perfect... with the combination of reindex data with a scheduler would help in this case? I just was not sure if reindex can be done with the destination on the same index all the time.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 8, 2020, 8:30am UTC](https://discuss.elastic.co/t/extract-a-subset-of-an-big-index/236360/6 "2020-07-08T08:30:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.