# I want to split the existing index to daily index

**URL:** <https://discuss.elastic.co/t/i-want-to-split-the-existing-index-to-daily-index/260501>\
**Category:** Elasticsearch\
**Created:** [January 8, 2021, 6:58am UTC](https://discuss.elastic.co/t/i-want-to-split-the-existing-index-to-daily-index/260501 "2021-01-08T06:58:10Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rohit\_Kumbhar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohit_kumbhar/32/67682_2.png) [@Rohit\_Kumbhar](https://discuss.elastic.co/u/Rohit_Kumbhar)\
**Post date:** [January 8, 2021, 6:58am UTC](https://discuss.elastic.co/t/i-want-to-split-the-existing-index-to-daily-index/260501/1 "2021-01-08T06:58:10Z")

</div>

Hello,

I want to split the existing index of size 200gb into daily index, That index has data of 1 month.  
I have setup of 8 core cpu and 16gb ram.  
what query should I write on dev tools or is there any other way to execute this.  
and how much time will it take to split the index.

I have tried this using logstash.  
I took input as elasticsearch index and gave output according to the daily index.  
but it takes a lot of time around 20 hours to split the 8 gb index after this the total size of daily indices increased more than 8 gb.

Please provide solution for this activity.  
Kindly help.  
Regards Rohit

---

<div class="post-metadata">

**Author:** ![myspacebarisbroken](https://avatars.discourse-cdn.com/v4/letter/m/e9a140/32.png) [@myspacebarisbroken](https://discuss.elastic.co/u/myspacebarisbroken)\
**Post date:** [January 9, 2021, 7:15am UTC](https://discuss.elastic.co/t/i-want-to-split-the-existing-index-to-daily-index/260501/2 "2021-01-09T07:15:52Z")

</div>

Hi Rohit,

I can't answer the first part, but you can use the rollover API to create daily a new daily index when conditions are triggered: [https://www.elastic.co/guide/en/elasticsearch/reference/master/indices-rollover-index.html](https://www.elastic.co/guide/en/elasticsearch/reference/master/indices-rollover-index.html)

But you could try creating a new index, then using the reindex API on certain date ranges and so forth.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [January 11, 2021, 12:14am UTC](https://discuss.elastic.co/t/i-want-to-split-the-existing-index-to-daily-index/260501/3 "2021-01-11T00:14:52Z")

</div>

Try using the `_reindex` API in Elasticsearch, with a timerange query to split things out.

---

<div class="post-metadata">

**Author:** ![Rohit\_Kumbhar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohit_kumbhar/32/67682_2.png) [@Rohit\_Kumbhar](https://discuss.elastic.co/u/Rohit_Kumbhar)\
**Post date:** [January 12, 2021, 6:00pm UTC](https://discuss.elastic.co/t/i-want-to-split-the-existing-index-to-daily-index/260501/4 "2021-01-12T18:00:48Z")

</div>

Thank you for your response,

I tried \_reindex API Already but it also takes too much time to create index for this huge data

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [January 12, 2021, 6:16pm UTC](https://discuss.elastic.co/t/i-want-to-split-the-existing-index-to-daily-index/260501/5 "2021-01-12T18:16:41Z")

</div>

Why are you looking to split the index? Is it due to query performance?

Which version of Elasticsearch are you using?

How many primary and replica shards does the index have?

---

<div class="post-metadata">

**Author:** ![ylasri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ylasri/32/86120_2.png) [@ylasri](https://discuss.elastic.co/u/ylasri)\
**Post date:** [January 12, 2021, 6:56pm UTC](https://discuss.elastic.co/t/i-want-to-split-the-existing-index-to-daily-index/260501/6 "2021-01-12T18:56:44Z")

</div>

You can use \_reindex with a painless script to change dynamically the index name based on the date of the event, something like this should work

```auto
POST _reindex?wait_for_completion=false
{
  "source": {
    "index": "source-index-name"
  },
  "dest": {
    "index": "destination-index-name"
  },
  "script": {
    "source": """
    
        def inputFormat = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss");
        def myDate = inputFormat.parse(ctx._source['@timestamp']);
        
        def outputFormat = new SimpleDateFormat("yyyy-MM-dd");
        def outputDay = outputFormat.format(myDate);
        
        ctx._index = "destination-index-name-" + outputDay;
"""
  }
}

```

---

<div class="post-metadata">

**Author:** ![ylasri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ylasri/32/86120_2.png) [@ylasri](https://discuss.elastic.co/u/ylasri)\
**Post date:** [January 12, 2021, 8:08pm UTC](https://discuss.elastic.co/t/i-want-to-split-the-existing-index-to-daily-index/260501/7 "2021-01-12T20:08:55Z")

</div>

You can also explore this processor inside an ingest pipeline when reindexing

> **[Date index name processor | Elasticsearch Reference \[7.10\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/date-index-name-processor.html)**

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 9, 2021, 8:08pm UTC](https://discuss.elastic.co/t/i-want-to-split-the-existing-index-to-daily-index/260501/8 "2021-02-09T20:08:55Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.