# Performance issue when using percolators **URL:** \ **Category:** Elasticsearch\ **Created:** [May 9, 2012, 7:55am UTC](https://discuss.elastic.co/t/performance-issue-when-using-percolators/7611 "2012-05-09T07:55:56Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Brett\_Anderson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brett_anderson/32/2555_2.png) [@Brett\_Anderson](https://discuss.elastic.co/u/Brett_Anderson)\ **Post date:** [May 9, 2012, 7:55am UTC](https://discuss.elastic.co/t/performance-issue-when-using-percolators/7611/1 "2012-05-09T07:55:56Z")
I'm using the percolator feature in ES to filter documents as they arrive in real-time. This filter can receive instant requests to modify the set of percolators as keywords change but it also deletes the index holding the percolators every hour. After deletion the index is re-created and the percolators are built from scratch again. This is to ensure that the percolators never go out of sync with the master list of keywords for very long. I've found that the percolators are able to process around 1.5K documents per second. However, after I delete and re-create the index as described above the rate of throughput drops to 500 ps, almost 30% of the original rate. If I re-start ES and then run the exact same deletion and re-creation the rate goes back to 1.5K (full speed). Is there any reason why deleting and recreating an index and by extension its percolators would cause a dip in performance? Again, restarting ES and then deleting and re-creating solves the problem. If needed I can try and create a gist to demonstrate this, but it obviously wouldn't be straight forward so I thought I'd ask first in-case there's a simple answer. Thanks, LJ. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [May 9, 2012, 10:48am UTC](https://discuss.elastic.co/t/performance-issue-when-using-percolators/7611/2 "2012-05-09T10:48:22Z")
Thats strange, it does not really matter. Can you list the order of operations that you do explicitly? (sample curl like deleting the index, and so on). On Wed, May 9, 2012 at 10:55 AM, Laser Jesus [brett.anderson.ftw@gmail.com](mailto:brett.anderson.ftw@gmail.com)wrote: > I'm using the percolator feature in ES to filter documents as they arrive > in real-time. This filter can receive instant requests to modify the set of > percolators as keywords change but it also deletes the index holding the > percolators every hour. After deletion the index is re-created and the > percolators are built from scratch again. This is to ensure that the > percolators never go out of sync with the master list of keywords for very > long. > > I've found that the percolators are able to process around 1.5K documents > per second. However, after I delete and re-create the index as described > above the rate of throughput drops to 500 ps, almost 30% of the original > rate. If I re-start ES and then run the exact same deletion and re-creation > the rate goes back to 1.5K (full speed). > > Is there any reason why deleting and recreating an index and by extension > its percolators would cause a dip in performance? Again, restarting ES and > then deleting and re-creating solves the problem. If needed I can try and > create a gist to demonstrate this, but it obviously wouldn't be straight > forward so I thought I'd ask first in-case there's a simple answer. > > Thanks, > LJ. ---
**Author:** ![Andrew\_at\_DataFeedFi](https://avatars.discourse-cdn.com/v4/letter/a/0ea827/32.png) [@Andrew\_at\_DataFeedFi](https://discuss.elastic.co/u/Andrew_at_DataFeedFi)\ **Post date:** [May 9, 2012, 2:43pm UTC](https://discuss.elastic.co/t/performance-issue-when-using-percolators/7611/3 "2012-05-09T14:43:05Z")
How many node & shards do you have for this index? Have you looked to make sure ES is not doing rebalance shards shortly after you delete and recreate the index? Andrew On May 9, 5:48 am, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > Thats strange, it does not really matter. Can you list the order of > operations that you do explicitly? (sample curl like deleting the index, > and so on). > > On Wed, May 9, 2012 at 10:55 AM, Laser Jesus > [brett.anderson....@gmail.com](mailto:brett.anderson....@gmail.com)wrote: > > > I'm using the percolator feature in ES to filter documents as they arrive > > in real-time. This filter can receive instant requests to modify the set of > > percolators as keywords change but it also deletes the index holding the > > percolators every hour. After deletion the index is re-created and the > > percolators are built from scratch again. This is to ensure that the > > percolators never go out of sync with the master list of keywords for very > > long. > > > I've found that the percolators are able to process around 1.5K documents > > per second. However, after I delete and re-create the index as described > > above the rate of throughput drops to 500 ps, almost 30% of the original > > rate. If I re-start ES and then run the exact same deletion and re-creation > > the rate goes back to 1.5K (full speed). > > > Is there any reason why deleting and recreating an index and by extension > > its percolators would cause a dip in performance? Again, restarting ES and > > then deleting and re-creating solves the problem. If needed I can try and > > create a gist to demonstrate this, but it obviously wouldn't be straight > > forward so I thought I'd ask first in-case there's a simple answer. > > > Thanks, > > LJ. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:29am UTC](https://discuss.elastic.co/t/performance-issue-when-using-percolators/7611/4 "2017-07-06T03:29:36Z")