# Cluster extremely slow after many bulk indexes **URL:** \ **Category:** Elasticsearch\ **Created:** [May 26, 2013, 5:17pm UTC](https://discuss.elastic.co/t/cluster-extremely-slow-after-many-bulk-indexes/12129 "2013-05-26T17:17:47Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![Christopher\_J\_Bottar](https://avatars.discourse-cdn.com/v4/letter/c/e0b2c6/32.png) [@Christopher\_J\_Bottar](https://discuss.elastic.co/u/Christopher_J_Bottar)\ **Post date:** [May 26, 2013, 5:17pm UTC](https://discuss.elastic.co/t/cluster-extremely-slow-after-many-bulk-indexes/12129/1 "2013-05-26T17:17:47Z")
Hi, I have about 1500 processes, each issuing a bulk index request of 500 documents. These 1500 processes run in parallel (well, in batches of 600 or so), and they complete in just a couple of minutes. In essence, the Es cluster is being hit with 1500 bulk index requests, each with 500 documents, over the course of a few minutes. First issue is that not all the bulk index requests return a proper response. This may be a client and/or proxy issue. I opened a ticket with Tire: [https://github.com/karmi/tire/issues/740](https://github.com/karmi/tire/issues/740) The second problem is that after issuing all these bulk requests, the Es cluster becomes very slow respond to requests. For instance, doing a search simply to query the total number of documents in the index, takes up to 18 seconds. My setup is a 2 node cluster: 30 shards, 0 replicas. Each node has 8 processors and 7.5 GB of memory. While the bulk indexing is going on, the machines' cpu doesn't go above 60% utilization. The load doesn't go above 3. In other words, the machines look like they aren't even breaking a sweat. Why does the cluster become unresponsive (err, very slow) then? Even after the cpu utilization drops significantly ([http://d.pr/i/PZGg](http://d.pr/i/PZGg)), a simple search (curl -X GET '[http://localhost:9200/unt\_production/\_search?pretty](http://localhost:9200/unt_production/_search?pretty)') still takes 17 seconds to complete. Thanks for the help. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![radu\_gheorghe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/radu_gheorghe/32/556_2.png) [@radu\_gheorghe](https://discuss.elastic.co/u/radu_gheorghe)\ **Post date:** [May 28, 2013, 10:39am UTC](https://discuss.elastic.co/t/cluster-extremely-slow-after-many-bulk-indexes/12129/2 "2013-05-28T10:39:52Z")
Hi Christopher, This situation might be caused by merges: > **[Elastic — The Search AI Company](https://www.elastic.co)** > > Power insights and outcomes with The Elastic Search AI Platform. See into your data and find answers that matter with enterprise solutions designed to help you accelerate time to insight. Try Elastic ... They're asynchronous, cause I/O load and invalidate caches. Warmers might help the cause: > **[Elastic — The Search AI Company](https://www.elastic.co)** > > Power insights and outcomes with The Elastic Search AI Platform. See into your data and find answers that matter with enterprise solutions designed to help you accelerate time to insight. Try Elastic ... Since you seem to have bursts of indexing, I would disable the warmers doing that, and enable them back once indexing is done. Otherwise, warmers will slow down your indexing. ## Best regards, Radu [http://sematext.com/](http://sematext.com/) -- Elasticsearch -- Solr -- Lucene On Sun, 26 May 2013 10:17:47 -0700 (PDT) "Christopher J. Bottaro" [cjbottaro@gmail.com](mailto:cjbottaro@gmail.com) wrote: > Hi, > > I have about 1500 processes, each issuing a bulk index request of 500 > documents. These 1500 processes run in parallel (well, in batches of > 600 or so), and they complete in just a couple of minutes. In > essence, the Es cluster is being hit with 1500 bulk index requests, > each with 500 documents, over the course of a few minutes. > > First issue is that not all the bulk index requests return a proper > response. This may be a client and/or proxy issue. I opened a > ticket with Tire: [bulk response is sometimes nil · Issue #740 · karmi/retire · GitHub](https://github.com/karmi/tire/issues/740) > > The second problem is that after issuing all these bulk requests, the > Es cluster becomes very slow respond to requests. For instance, > doing a search simply to query the total number of documents in the > index, takes up to 18 seconds. > > My setup is a 2 node cluster: 30 shards, 0 replicas. Each node has > 8 processors and 7.5 GB of memory. While the bulk indexing is going > on, the machines' cpu doesn't go above 60% utilization. The load > doesn't go above 3. In other words, the machines look like they > aren't even breaking a sweat. > > Why does the cluster become unresponsive (err, very slow) then? > > Even after the cpu utilization drops significantly > ([http://d.pr/i/PZGg](http://d.pr/i/PZGg)), a simple search (curl -X GET > '[http://localhost:9200/unt\_production/\_search?pretty](http://localhost:9200/unt_production/_search?pretty)') still takes 17 > seconds to complete. > > Thanks for the help. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:34am UTC](https://discuss.elastic.co/t/cluster-extremely-slow-after-many-bulk-indexes/12129/3 "2017-07-06T02:34:26Z")