# How to safely clean old documents (by date) **URL:** \ **Category:** Elasticsearch\ **Created:** [May 3, 2015, 8:24pm UTC](https://discuss.elastic.co/t/how-to-safely-clean-old-documents-by-date/23523 "2015-05-03T20:24:00Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![AALISHE](https://avatars.discourse-cdn.com/v4/letter/a/3e96dc/32.png) [@AALISHE](https://discuss.elastic.co/u/AALISHE)\ **Post date:** [May 3, 2015, 8:24pm UTC](https://discuss.elastic.co/t/how-to-safely-clean-old-documents-by-date/23523/1 "2015-05-03T20:24:00Z")
Hi, I have ES "0.20.3" .. with a single index replicated on 2 servers .. with 5 shards ..\* size: 57.2gb / docs: 36060297 \* I have webpages(the docs) indexed since 2013 ... so I want to delete everything and keep 1 year worth of documents how can I do this safely on a production setup am thinking of the following: 1- make a copy of the current index ... put it next to it (with a different name ofcourse) ... _(how do I accomplish this)_ 2- delete documents before May 2014 from the copied index 3- Rename the old index and leave / Delete it OR 1- make an empty index 2- pull documents after May2014 from the current index to the new one \*(how do I accomplish this)\*3- Rename the old index and leave / Delete it appreciate your help guys... thanks! -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f420f325-9f60-49d9-a9f6-d56528f32a99%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f420f325-9f60-49d9-a9f6-d56528f32a99%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [May 3, 2015, 9:14pm UTC](https://discuss.elastic.co/t/how-to-safely-clean-old-documents-by-date/23523/2 "2015-05-03T21:14:59Z")
Definitely the second option. Use scan and scroll (search for reindex on the website). Instead of renaming, I would use aliases and switch the alias from old to new index. Then close or remove the old index. -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > Le 3 mai 2015 à 22:24, AALISHE [aalishe@gmail.com](mailto:aalishe@gmail.com) a écrit : > > Hi, > > I have ES "0.20.3" .. with a single index replicated on 2 servers .. with 5 shards .. size: 57.2gb / docs: 36060297 > > I have webpages(the docs) indexed since 2013 ... so I want to delete everything and keep 1 year worth of documents > > how can I do this safely on a production setup > > am thinking of the following: > > 1- make a copy of the current index ... put it next to it (with a different name ofcourse) ... (how do I accomplish this) > 2- delete documents before May 2014 from the copied index > 3- Rename the old index and leave / Delete it > > OR > > 1- make an empty index > 2- pull documents after May2014 from the current index to the new one (how do I accomplish this) > 3- Rename the old index and leave / Delete it > > ## appreciate your help guys... thanks! > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f420f325-9f60-49d9-a9f6-d56528f32a99%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f420f325-9f60-49d9-a9f6-d56528f32a99%40googlegroups.com). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/28864A70-C04D-4845-AF22-7C5EDCB87FB2%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/28864A70-C04D-4845-AF22-7C5EDCB87FB2%40pilato.fr). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![AALISHE](https://avatars.discourse-cdn.com/v4/letter/a/3e96dc/32.png) [@AALISHE](https://discuss.elastic.co/u/AALISHE)\ **Post date:** [May 3, 2015, 9:36pm UTC](https://discuss.elastic.co/t/how-to-safely-clean-old-documents-by-date/23523/3 "2015-05-03T21:36:10Z")
Thanks David! do you know how I perform step (2) _pull documents after May2014 from the current index to the new one_ On Monday, May 4, 2015 at 12:15:22 AM UTC+3, David Pilato wrote: > Definitely the second option. > Use scan and scroll (search for reindex on the website). > > Instead of renaming, I would use aliases and switch the alias from old to > new index. > > Then close or remove the old index. > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 3 mai 2015 à 22:24, AALISHE \<[aal...@gmail.com](mailto:aal...@gmail.com) \\> a écrit : > > Hi, > > I have ES "0.20.3" .. with a single index replicated on 2 servers .. with > 5 shards ..\* size: 57.2gb / docs: 36060297 \* > > I have webpages(the docs) indexed since 2013 ... so I want to delete > everything and keep 1 year worth of documents > > how can I do this safely on a production setup > > am thinking of the following: > > 1- make a copy of the current index ... put it next to it (with a > different name ofcourse) ... _(how do I accomplish this)_ > 2- delete documents before May 2014 from the copied index > 3- Rename the old index and leave / Delete it > > OR > > 1- make an empty index > 2- pull documents after May2014 from the current index to the new one > \*(how do I accomplish this)\*3- Rename the old index and leave / Delete > it > > appreciate your help guys... > thanks! > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/f420f325-9f60-49d9-a9f6-d56528f32a99%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f420f325-9f60-49d9-a9f6-d56528f32a99%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/f420f325-9f60-49d9-a9f6-d56528f32a99%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f420f325-9f60-49d9-a9f6-d56528f32a99%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/9c312789-2492-4bdf-a660-19d542df4977%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/9c312789-2492-4bdf-a660-19d542df4977%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [May 3, 2015, 10:06pm UTC](https://discuss.elastic.co/t/how-to-safely-clean-old-documents-by-date/23523/4 "2015-05-03T22:06:17Z")
Searching for reindex in docs would have directed you to [Reindexing Your Data | Elasticsearch: The Definitive Guide [2.x] | Elastic](http://www.elastic.co/guide/en/elasticsearch/guide/current/reindex.html) David > Le 3 mai 2015 à 23:36, AALISHE [aalishe@gmail.com](mailto:aalishe@gmail.com) a écrit : > > Thanks David! > > do you know how I perform step (2) pull documents after May2014 from the current index to the new one > > > On Monday, May 4, 2015 at 12:15:22 AM UTC+3, David Pilato wrote: > > Definitely the second option. > > Use scan and scroll (search for reindex on the website). > > > > Instead of renaming, I would use aliases and switch the alias from old to new index. > > > > Then close or remove the old index. > > > > -- > > David 😉 > > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > > > > Le 3 mai 2015 à 22:24, AALISHE [aal...@gmail.com](mailto:aal...@gmail.com) a écrit : > > > > > > Hi, > > > > > > I have ES "0.20.3" .. with a single index replicated on 2 servers .. with 5 shards .. size: 57.2gb / docs: 36060297 > > > > > > I have webpages(the docs) indexed since 2013 ... so I want to delete everything and keep 1 year worth of documents > > > > > > how can I do this safely on a production setup > > > > > > am thinking of the following: > > > > > > 1- make a copy of the current index ... put it next to it (with a different name ofcourse) ... (how do I accomplish this) > > > 2- delete documents before May 2014 from the copied index > > > 3- Rename the old index and leave / Delete it > > > > > > OR > > > > > > 1- make an empty index > > > 2- pull documents after May2014 from the current index to the new one (how do I accomplish this) > > > 3- Rename the old index and leave / Delete it > > > > > > ## appreciate your help guys... thanks! > > > > > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f420f325-9f60-49d9-a9f6-d56528f32a99%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f420f325-9f60-49d9-a9f6-d56528f32a99%40googlegroups.com). > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/9c312789-2492-4bdf-a660-19d542df4977%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/9c312789-2492-4bdf-a660-19d542df4977%40googlegroups.com). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/F64B5203-F4A7-4FB0-B34C-44F8D3249D8E%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/F64B5203-F4A7-4FB0-B34C-44F8D3249D8E%40pilato.fr). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [May 3, 2015, 11:09pm UTC](https://discuss.elastic.co/t/how-to-safely-clean-old-documents-by-date/23523/5 "2015-05-03T23:09:35Z")
Just a side note, if you are using time based data then it makes a lot of sense to use time based indices - ie daily, weekly, monthly. On 4 May 2015 at 08:06, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > Searching for reindex in docs would have directed you to > [Reindexing Your Data | Elasticsearch: The Definitive Guide [2.x] | Elastic](http://www.elastic.co/guide/en/elasticsearch/guide/current/reindex.html) > > David > > Le 3 mai 2015 à 23:36, AALISHE [aalishe@gmail.com](mailto:aalishe@gmail.com) a écrit : > > Thanks David! > > do you know how I perform step (2) _pull documents after May2014 from > the current index to the new one_ > > On Monday, May 4, 2015 at 12:15:22 AM UTC+3, David Pilato wrote: > > > Definitely the second option. > > Use scan and scroll (search for reindex on the website). > > > > Instead of renaming, I would use aliases and switch the alias from old to > > new index. > > > > Then close or remove the old index. > > > > -- > > David 😉 > > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > > > Le 3 mai 2015 à 22:24, AALISHE [aal...@gmail.com](mailto:aal...@gmail.com) a écrit : > > > > Hi, > > > > I have ES "0.20.3" .. with a single index replicated on 2 servers .. > > with 5 shards ..\* size: 57.2gb / docs: 36060297 \* > > > > I have webpages(the docs) indexed since 2013 ... so I want to delete > > everything and keep 1 year worth of documents > > > > how can I do this safely on a production setup > > > > am thinking of the following: > > > > 1- make a copy of the current index ... put it next to it (with a > > different name ofcourse) ... _(how do I accomplish this)_ > > 2- delete documents before May 2014 from the copied index > > 3- Rename the old index and leave / Delete it > > > > OR > > > > 1- make an empty index > > 2- pull documents after May2014 from the current index to the new one > > \*(how do I accomplish this)\*3- Rename the old index and leave / Delete > > it > > > > appreciate your help guys... > > thanks! > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/f420f325-9f60-49d9-a9f6-d56528f32a99%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f420f325-9f60-49d9-a9f6-d56528f32a99%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/f420f325-9f60-49d9-a9f6-d56528f32a99%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f420f325-9f60-49d9-a9f6-d56528f32a99%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/9c312789-2492-4bdf-a660-19d542df4977%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/9c312789-2492-4bdf-a660-19d542df4977%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/9c312789-2492-4bdf-a660-19d542df4977%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/9c312789-2492-4bdf-a660-19d542df4977%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/F64B5203-F4A7-4FB0-B34C-44F8D3249D8E%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/F64B5203-F4A7-4FB0-B34C-44F8D3249D8E%40pilato.fr) > [https://groups.google.com/d/msgid/elasticsearch/F64B5203-F4A7-4FB0-B34C-44F8D3249D8E%40pilato.fr?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/F64B5203-F4A7-4FB0-B34C-44F8D3249D8E%40pilato.fr?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_dBySiL84TBMqYxiu2odL52oBsB2s2cW4VqeocFxLmnA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_dBySiL84TBMqYxiu2odL52oBsB2s2cW4VqeocFxLmnA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:16am UTC](https://discuss.elastic.co/t/how-to-safely-clean-old-documents-by-date/23523/6 "2017-07-06T00:16:04Z")