# Reindexing a large collection into time based indices **URL:** \ **Category:** Elasticsearch\ **Created:** [October 2, 2016, 9:49pm UTC](https://discuss.elastic.co/t/reindexing-a-large-collection-into-time-based-indices/62016 "2016-10-02T21:49:48Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![yoitsro](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yoitsro/32/6192_2.png) [@yoitsro](https://discuss.elastic.co/u/yoitsro)\ **Post date:** [October 2, 2016, 9:49pm UTC](https://discuss.elastic.co/t/reindexing-a-large-collection-into-time-based-indices/62016/1 "2016-10-02T21:49:48Z")
Assuming I have a 30 shard index with over 200 million documents in it and I wanted to split these out into a time based index, how would I do this without affecting response times? The other issue is storage space, but I could easily scale up the instances before reindexing. Cheers, Ro. ---
**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\ **Post date:** [October 2, 2016, 10:20pm UTC](https://discuss.elastic.co/t/reindexing-a-large-collection-into-time-based-indices/62016/2 "2016-10-02T22:20:25Z")
200 million is usually fine.... Splitting it into smaller indexes will help if you can write your queries so they only target the indexes that contain the docs. In 5.0 we rewrite the queries on the target shards so that if an index doesn't have any docs in the time range then it becomes a match\_none so it is cheap. Anyway, yeah, your best bet is to reindex using the time ranges in the filter. I'd add more space to the cluster rather than try and juggle thing, delete-by-query isn't a good way to free space so you can't easily juggle the free space. ---
**Author:** ![yoitsro](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yoitsro/32/6192_2.png) [@yoitsro](https://discuss.elastic.co/u/yoitsro)\ **Post date:** [October 2, 2016, 10:23pm UTC](https://discuss.elastic.co/t/reindexing-a-large-collection-into-time-based-indices/62016/3 "2016-10-02T22:23:27Z")
Hey Nick, Thanks very much for this. The other issue is that the index is a live index with full read/write access across the index. How would I ensure there's no data loss? And wouldn't there be any latency increase across the cluster if I was reindexing the documents? ---
**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\ **Post date:** [October 2, 2016, 10:37pm UTC](https://discuss.elastic.co/t/reindexing-a-large-collection-into-time-based-indices/62016/4 "2016-10-02T22:37:11Z")
We don't really have a thing for live indexes. Sadly, that is a thing you'll have to work out. Do you have any restrictions on your access patterns? Sometimes that helps. You could have the index write to both, but that can be difficult depending. ---
**Author:** ![yoitsro](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yoitsro/32/6192_2.png) [@yoitsro](https://discuss.elastic.co/u/yoitsro)\ **Post date:** [October 2, 2016, 10:40pm UTC](https://discuss.elastic.co/t/reindexing-a-large-collection-into-time-based-indices/62016/5 "2016-10-02T22:40:02Z")
Ahh! That would be perfect actually! I think that's possible using the system we have. ---
**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\ **Post date:** [October 2, 2016, 10:51pm UTC](https://discuss.elastic.co/t/reindexing-a-large-collection-into-time-based-indices/62016/6 "2016-10-02T22:51:37Z")
Oh no! I mistyped. Misphoned. Something. ES doesn't have a thing to have the write forked to two indexes. Thatd be a thing you'd have to do in your application. Sorry! ---
**Author:** ![yoitsro](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yoitsro/32/6192_2.png) [@yoitsro](https://discuss.elastic.co/u/yoitsro)\ **Post date:** [October 4, 2016, 8:57am UTC](https://discuss.elastic.co/t/reindexing-a-large-collection-into-time-based-indices/62016/7 "2016-10-04T08:57:40Z")
No, that's all good actually. We can do this application side without much fuss. Thank you! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 10:15pm UTC](https://discuss.elastic.co/t/reindexing-a-large-collection-into-time-based-indices/62016/8 "2017-07-05T22:15:15Z")