# Elastic Search - How to delete old records and improve performance **URL:** \ **Category:** Elasticsearch\ **Created:** [March 30, 2015, 8:56pm UTC](https://discuss.elastic.co/t/elastic-search-how-to-delete-old-records-and-improve-performance/22992 "2015-03-30T20:56:25Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![AALISHE](https://avatars.discourse-cdn.com/v4/letter/a/3e96dc/32.png) [@AALISHE](https://discuss.elastic.co/u/AALISHE)\ **Post date:** [March 30, 2015, 8:56pm UTC](https://discuss.elastic.co/t/elastic-search-how-to-delete-old-records-and-improve-performance/22992/1 "2015-03-30T20:56:25Z")
Hi all, I have a production box running old ES "0.20.3" .. I need help in cleaning data back from 2012 ... because although I moved the whole indexs to a machine with 6 SSD drives ... queries got better .. but still I need to make this thing faster is there a \*safe \*way to clean (choose what to clean based on date) and improve the performance? cheers! -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAOG9y1VndKdaZTt19PN2aVobL5\_-02QctZ\_8t44VXuGG9azrqA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAOG9y1VndKdaZTt19PN2aVobL5_-02QctZ_8t44VXuGG9azrqA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![AALISHE](https://avatars.discourse-cdn.com/v4/letter/a/3e96dc/32.png) [@AALISHE](https://discuss.elastic.co/u/AALISHE)\ **Post date:** [March 31, 2015, 12:17am UTC](https://discuss.elastic.co/t/elastic-search-how-to-delete-old-records-and-improve-performance/22992/2 "2015-03-31T00:17:23Z")
anything?... On Monday, March 30, 2015 at 11:56:46 PM UTC+3, AALISHE wrote: > Hi all, > > I have a production box running old ES "0.20.3" .. I need help in > cleaning data back from 2012 ... because although I moved the whole indexs > to a machine with 6 SSD drives ... queries got better .. but still I need > to make this thing faster > > is there a \*safe \*way to clean (choose what to clean based on date) and > improve the performance? > > cheers! -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/e9d34bef-f899-40e5-82d6-8b74b3adf8d4%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e9d34bef-f899-40e5-82d6-8b74b3adf8d4%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [March 31, 2015, 1:00am UTC](https://discuss.elastic.co/t/elastic-search-how-to-delete-old-records-and-improve-performance/22992/3 "2015-03-31T01:00:10Z")
This is a community based list and support is provided by volunteers as their time permits. If you want timely responses then I'd recommend you sign up for some support, we'd be happy to have you as a customer [Subscriptions | Elastic Stack Products & Support | Elastic](https://www.elastic.co/subscriptions/) ๐Ÿ™‚ Is all the data in the one index, or are they time based? If you need ES to be faster then I'd suggest you upgrade, 0.20 is no longer supported and there are a _lot_ of performance improvements in later versions. On 31 March 2015 at 11:17, AALISHE [aalishe@gmail.com](mailto:aalishe@gmail.com) wrote: > anything?... > > On Monday, March 30, 2015 at 11:56:46 PM UTC+3, AALISHE wrote: > > > Hi all, > > > > I have a production box running old ES "0.20.3" .. I need help in > > cleaning data back from 2012 ... because although I moved the whole indexs > > to a machine with 6 SSD drives ... queries got better .. but still I need > > to make this thing faster > > > > is there a \*safe \*way to clean (choose what to clean based on date) and > > improve the performance? > > > > cheers! > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/e9d34bef-f899-40e5-82d6-8b74b3adf8d4%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e9d34bef-f899-40e5-82d6-8b74b3adf8d4%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/e9d34bef-f899-40e5-82d6-8b74b3adf8d4%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/e9d34bef-f899-40e5-82d6-8b74b3adf8d4%40googlegroups.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-Y3SF%2BwO%2BQMrUpc3Aj46Fx85xRyRpBMqcQ-Xxe0tLQdg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-Y3SF%2BwO%2BQMrUpc3Aj46Fx85xRyRpBMqcQ-Xxe0tLQdg%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![AALISHE](https://avatars.discourse-cdn.com/v4/letter/a/3e96dc/32.png) [@AALISHE](https://discuss.elastic.co/u/AALISHE)\ **Post date:** [March 31, 2015, 6:38am UTC](https://discuss.elastic.co/t/elastic-search-how-to-delete-old-records-and-improve-performance/22992/4 "2015-03-31T06:38:43Z")
Hi Mark, thanks for taking the time to reply 1- yes I believe all the data is in one index .. I had attached the index\_status output ... and a screenshot for the head\_plugin output 2- I am considering the upgrade to latest version .. but would that impose a risk (if any)? thanks again cheers! -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/d5529d52-0a21-402a-8265-78da41b2b05f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d5529d52-0a21-402a-8265-78da41b2b05f%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Mano\_Ksoin](https://avatars.discourse-cdn.com/v4/letter/m/ea5d25/32.png) [@Mano\_Ksoin](https://discuss.elastic.co/u/Mano_Ksoin)\ **Post date:** [March 31, 2015, 1:42pm UTC](https://discuss.elastic.co/t/elastic-search-how-to-delete-old-records-and-improve-performance/22992/5 "2015-03-31T13:42:26Z")
Can you create a new index and exclude what you dont need. we did this recently because of some other mapping reason. take a snapshot (with newwer version) before doing so. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/c473d836-50dd-4706-b7ab-da29cacd2612%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c473d836-50dd-4706-b7ab-da29cacd2612%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![AALISHE](https://avatars.discourse-cdn.com/v4/letter/a/3e96dc/32.png) [@AALISHE](https://discuss.elastic.co/u/AALISHE)\ **Post date:** [March 31, 2015, 2:48pm UTC](https://discuss.elastic.co/t/elastic-search-how-to-delete-old-records-and-improve-performance/22992/6 "2015-03-31T14:48:39Z")
mkBig ... thanks for the suggestion ... but how do I exclude the things I dont need ? cheers! -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/454830bc-a9c5-4803-ae95-8b37f65fedbf%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/454830bc-a9c5-4803-ae95-8b37f65fedbf%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![AALISHE](https://avatars.discourse-cdn.com/v4/letter/a/3e96dc/32.png) [@AALISHE](https://discuss.elastic.co/u/AALISHE)\ **Post date:** [March 31, 2015, 8:13pm UTC](https://discuss.elastic.co/t/elastic-search-how-to-delete-old-records-and-improve-performance/22992/7 "2015-03-31T20:13:07Z")
mkBig ... thanks for the suggestion ... but how do I exclude the things I dont need ? cheers! -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/64a09808-e8f6-4588-959b-8855a851ed03%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/64a09808-e8f6-4588-959b-8855a851ed03%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![eperry](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eperry/32/551_2.png) [@eperry](https://discuss.elastic.co/u/eperry)\ **Post date:** [April 1, 2015, 2:59am UTC](https://discuss.elastic.co/t/elastic-search-how-to-delete-old-records-and-improve-performance/22992/8 "2015-04-01T02:59:32Z")
You could do a few things, from my perspective. (Hope this hits some idea's you like) - upgrade, never hurts ๐Ÿ™‚ as long as you read the release notes to make sure nothing your depending on. - Add some filters to your existing queries to exclude unneeded data - ## you can create "filter aliases" which lets you set up predefined alias with filtered terms in-case you can't delete or loose data [Index Aliases | Elasticsearch Guide [1.3] | Elastic](http://www.elastic.co/guide/en/elasticsearch/reference/1.3/indices-aliases.html) - I would rotate your index so all new data would go in to a new one while the archive can wither and die - Maybe rotate it on a daily basis or a point of time that works for your data sets you can still have basic aliases to summarize multiple days depending on your query needs - Use a project called curator to purge old data on a routine basis (cron) - [Home ยท elastic/curator Wiki ยท GitHub](https://github.com/elastic/curator/wiki) - Finally if you need to massage the data you can look at this project - [GitHub - elasticsearch-dump/elasticsearch-dump: Import and export tools for elasticsearch & opensearch](https://github.com/taskrabbit/elasticsearch-dump) it does full dumps and also Query Style dumpes. - Dump the whole index - Then dump with a query - Delete and import your query dump to a Date based index that can be purged later by curator - Oh just thought of this one, increase the number of shards and ES nodes, I see you have only 2 - To do this will require you to export and import the data again. (at least for the old data On Tuesday, March 31, 2015 at 4:13:07 PM UTC-4, AALISHE wrote: > mkBig ... thanks for the suggestion ... but how do I exclude the things I > dont need ? > > cheers! -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/4228dbd3-84a1-41ba-bed9-3bcf4cb70033%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/4228dbd3-84a1-41ba-bed9-3bcf4cb70033%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:22am UTC](https://discuss.elastic.co/t/elastic-search-how-to-delete-old-records-and-improve-performance/22992/9 "2017-07-06T00:22:29Z")