# Can't execute bulk query **URL:** \ **Category:** Elasticsearch\ **Created:** [July 1, 2011, 4:49pm UTC](https://discuss.elastic.co/t/cant-execute-bulk-query/4750 "2011-07-01T16:49:24Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Dan\_Diephouse](https://avatars.discourse-cdn.com/v4/letter/d/779978/32.png) [@Dan\_Diephouse](https://discuss.elastic.co/u/Dan_Diephouse)\ **Post date:** [July 1, 2011, 4:49pm UTC](https://discuss.elastic.co/t/cant-execute-bulk-query/4750/1 "2011-07-01T16:49:24Z")
I'm trying to execute a query to clean out some stuff from our database, but it's failing. This same query works on another ES instance that we have but without so much data in it. Any idea why I'm getting this failure or where I look for more information? curl -XDELETE '[http://host:9200/ion/dantest/\_query](http://host:9200/ion/dantest/_query)? [q=fields.domain=dantest.muleion.com](http://q=fields.domain=dantest.muleion.com)' {"ok":true,"\_indices":{"ion":{"\_shards":{"total":5,"successful": 4,"failed":1}}}} Cheers, Dan ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [July 1, 2011, 4:55pm UTC](https://discuss.elastic.co/t/cant-execute-bulk-query/4750/2 "2011-07-01T16:55:35Z")
It failed on one shard, anything in the logs? On Friday, July 1, 2011 at 7:49 PM, Dan Diephouse wrote: > I'm trying to execute a query to clean out some stuff from our > database, but it's failing. This same query works on another ES > instance that we have but without so much data in it. Any idea why I'm > getting this failure or where I look for more information? > > curl -XDELETE '[http://host:9200/ion/dantest/\_query](http://host:9200/ion/dantest/_query)? > [q=fields.domain=dantest.muleion.com](http://q=fields.domain=dantest.muleion.com) ([http://dantest.muleion.com](http://dantest.muleion.com))' > {"ok":true,"\_indices":{"ion":{"\_shards":{"total":5,"successful": > 4,"failed":1}}}} > > Cheers, > Dan ---
**Author:** ![Dan\_Diephouse](https://avatars.discourse-cdn.com/v4/letter/d/779978/32.png) [@Dan\_Diephouse](https://discuss.elastic.co/u/Dan_Diephouse)\ **Post date:** [July 1, 2011, 6:27pm UTC](https://discuss.elastic.co/t/cant-execute-bulk-query/4750/3 "2011-07-01T18:27:59Z")
Command: # date && curl -XDELETE '[http://host:9200/ion/dantest/\_query](http://host:9200/ion/dantest/_query)? [q=fields.domain=dantest.muleion.com](http://q=fields.domain=dantest.muleion.com)' && echo && date Fri Jul 1 14:25:32 EDT 2011 {"ok":true,"\_indices":{"ion":{"\_shards":{"total":5,"successful": 4,"failed":1}}}} Fri Jul 1 14:26:35 EDT 2011 Logs: [2011-07-01 14:25:32,792][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][103] took [6ms]/[1.4s], reclaimed [32.6mb], leaving [223.2mb] used, max [4.1gb] [2011-07-01 14:25:33,794][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][104] took [7ms]/[1.4s], reclaimed [33.1mb], leaving [223.4mb] used, max [4.1gb] [2011-07-01 14:25:34,794][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][105] took [6ms]/[1.4s], reclaimed [32.4mb], leaving [224.3mb] used, max [4.1gb] [2011-07-01 14:25:35,819][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][106] took [6ms]/[1.4s], reclaimed [33mb], leaving [224.6mb] used, max [4.1gb] [2011-07-01 14:25:36,821][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][108] took [9ms]/[1.4s], reclaimed [32.6mb], leaving [226.2mb] used, max [4.1gb] [2011-07-01 14:25:38,101][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][110] took [11ms]/[1.5s], reclaimed [32.2mb], leaving [227.5mb] used, max [4.1gb] [2011-07-01 14:25:39,103][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][112] took [7ms]/[1.5s], reclaimed [32.9mb], leaving [228.3mb] used, max [4.1gb] [2011-07-01 14:25:40,105][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][113] took [5ms]/[1.5s], reclaimed [32.9mb], leaving [228.6mb] used, max [4.1gb] [2011-07-01 14:25:41,107][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][114] took [6ms]/[1.5s], reclaimed [33.3mb], leaving [228.5mb] used, max [4.1gb] [2011-07-01 14:25:43,111][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][116] took [6ms]/[1.6s], reclaimed [32.7mb], leaving [229.9mb] used, max [4.1gb] [2011-07-01 14:25:46,171][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][117] took [6ms]/[1.6s], reclaimed [32.6mb], leaving [230.5mb] used, max [4.1gb] [2011-07-01 14:25:47,173][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][118] took [6ms]/[1.6s], reclaimed [32.8mb], leaving [231mb] used, max [4.1gb] [2011-07-01 14:25:48,175][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][119] took [21ms]/[1.6s], reclaimed [32.9mb], leaving [231.4mb] used, max [4.1gb] [2011-07-01 14:25:50,209][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][120] took [65ms]/[1.7s], reclaimed [33mb], leaving [231.6mb] used, max [4.1gb] [2011-07-01 14:25:51,211][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][121] took [8ms]/[1.7s], reclaimed [32.4mb], leaving [232.5mb] used, max [4.1gb] [2011-07-01 14:25:54,216][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][122] took [9ms]/[1.7s], reclaimed [33.3mb], leaving [232.4mb] used, max [4.1gb] [2011-07-01 14:25:57,219][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][123] took [7ms]/[1.7s], reclaimed [33mb], leaving [232.7mb] used, max [4.1gb] [2011-07-01 14:26:00,224][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][124] took [3ms]/[1.7s], reclaimed [33.2mb], leaving [232.8mb] used, max [4.1gb] [2011-07-01 14:26:02,280][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][125] took [4ms]/[1.7s], reclaimed [33.4mb], leaving [232.6mb] used, max [4.1gb] [2011-07-01 14:26:05,324][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][126] took [4ms]/[1.7s], reclaimed [33.3mb], leaving [232.6mb] used, max [4.1gb] [2011-07-01 14:26:09,403][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][127] took [3ms]/[1.7s], reclaimed [33mb], leaving [232.9mb] used, max [4.1gb] [2011-07-01 14:26:12,451][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][128] took [3ms]/[1.7s], reclaimed [33.3mb], leaving [232.8mb] used, max [4.1gb] [2011-07-01 14:26:14,454][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][129] took [3ms]/[1.7s], reclaimed [33.4mb], leaving [232.7mb] used, max [4.1gb] [2011-07-01 14:26:17,461][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][130] took [3ms]/[1.7s], reclaimed [33.1mb], leaving [232.9mb] used, max [4.1gb] [2011-07-01 14:26:20,491][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][131] took [4ms]/[1.7s], reclaimed [33.4mb], leaving [232.7mb] used, max [4.1gb] [2011-07-01 14:26:24,544][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][132] took [4ms]/[1.7s], reclaimed [33.1mb], leaving [232.9mb] used, max [4.1gb] [2011-07-01 14:26:26,547][DEBUG][monitor.jvm] [Jenkins, Abner] [gc][ParNew][133] took [3ms]/[1.7s], reclaimed [33.2mb], leaving [232.9mb] used, max [4.1gb] This is elasticsearch on a single host. No fancy setup, pretty much straight out of the box, but with 4 GB of memory. Dan On Jul 1, 12:55 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > It failed on one shard, anything in the logs? > > On Friday, July 1, 2011 at 7:49 PM, Dan Diephouse wrote: > > > I'm trying to execute a query to clean out some stuff from our > > database, but it's failing. This same query works on another ES > > instance that we have but without so much data in it. Any idea why I'm > > getting this failure or where I look for more information? > > > curl -XDELETE '[http://host:9200/ion/dantest/\_query](http://host:9200/ion/dantest/_query)? > > [q=fields.domain=dantest.muleion.com](http://q=fields.domain=dantest.muleion.com) ([http://dantest.muleion.com](http://dantest.muleion.com))' > > {"ok":true,"\_indices":{"ion":{"\_shards":{"total":5,"successful": > > 4,"failed":1}}}} > > > Cheers, > > Dan ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [July 2, 2011, 8:44pm UTC](https://discuss.elastic.co/t/cant-execute-bulk-query/4750/4 "2011-07-02T20:44:18Z")
String..., when theres a failure, then it should be logged. The problem with the delete\_by\_query API is that it does not return a message of the failure (as opposed to other "broadcast" APIs, like search) ... . Anything I can try and recreate? Don't mind getting hte data dir if its test data, and I can try and repro it locally. On Friday, July 1, 2011 at 9:27 PM, Dan Diephouse wrote: > Command: > > # date && curl -XDELETE '[http://host:9200/ion/dantest/\_query](http://host:9200/ion/dantest/_query)? > > [q=fields.domain=dantest.muleion.com](http://q=fields.domain=dantest.muleion.com) ([http://dantest.muleion.com](http://dantest.muleion.com))' > && echo && date > Fri Jul 1 14:25:32 EDT 2011 > {"ok":true,"\_indices":{"ion":{"\_shards":{"total":5,"successful": > 4,"failed":1}}}} > Fri Jul 1 14:26:35 EDT 2011 > > Logs: > > [2011-07-01 14:25:32,792][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][103] took [6ms]/[1.4s], reclaimed [32.6mb], > leaving [223.2mb] used, max [4.1gb] > [2011-07-01 14:25:33,794][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][104] took [7ms]/[1.4s], reclaimed [33.1mb], > leaving [223.4mb] used, max [4.1gb] > [2011-07-01 14:25:34,794][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][105] took [6ms]/[1.4s], reclaimed [32.4mb], > leaving [224.3mb] used, max [4.1gb] > [2011-07-01 14:25:35,819][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][106] took [6ms]/[1.4s], reclaimed [33mb], leaving > [224.6mb] used, max [4.1gb] > [2011-07-01 14:25:36,821][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][108] took [9ms]/[1.4s], reclaimed [32.6mb], > leaving [226.2mb] used, max [4.1gb] > [2011-07-01 14:25:38,101][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][110] took [11ms]/[1.5s], reclaimed [32.2mb], > leaving [227.5mb] used, max [4.1gb] > [2011-07-01 14:25:39,103][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][112] took [7ms]/[1.5s], reclaimed [32.9mb], > leaving [228.3mb] used, max [4.1gb] > [2011-07-01 14:25:40,105][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][113] took [5ms]/[1.5s], reclaimed [32.9mb], > leaving [228.6mb] used, max [4.1gb] > [2011-07-01 14:25:41,107][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][114] took [6ms]/[1.5s], reclaimed [33.3mb], > leaving [228.5mb] used, max [4.1gb] > [2011-07-01 14:25:43,111][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][116] took [6ms]/[1.6s], reclaimed [32.7mb], > leaving [229.9mb] used, max [4.1gb] > [2011-07-01 14:25:46,171][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][117] took [6ms]/[1.6s], reclaimed [32.6mb], > leaving [230.5mb] used, max [4.1gb] > [2011-07-01 14:25:47,173][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][118] took [6ms]/[1.6s], reclaimed [32.8mb], > leaving [231mb] used, max [4.1gb] > [2011-07-01 14:25:48,175][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][119] took [21ms]/[1.6s], reclaimed [32.9mb], > leaving [231.4mb] used, max [4.1gb] > [2011-07-01 14:25:50,209][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][120] took [65ms]/[1.7s], reclaimed [33mb], leaving > [231.6mb] used, max [4.1gb] > [2011-07-01 14:25:51,211][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][121] took [8ms]/[1.7s], reclaimed [32.4mb], > leaving [232.5mb] used, max [4.1gb] > [2011-07-01 14:25:54,216][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][122] took [9ms]/[1.7s], reclaimed [33.3mb], > leaving [232.4mb] used, max [4.1gb] > [2011-07-01 14:25:57,219][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][123] took [7ms]/[1.7s], reclaimed [33mb], leaving > [232.7mb] used, max [4.1gb] > [2011-07-01 14:26:00,224][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][124] took [3ms]/[1.7s], reclaimed [33.2mb], > leaving [232.8mb] used, max [4.1gb] > [2011-07-01 14:26:02,280][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][125] took [4ms]/[1.7s], reclaimed [33.4mb], > leaving [232.6mb] used, max [4.1gb] > [2011-07-01 14:26:05,324][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][126] took [4ms]/[1.7s], reclaimed [33.3mb], > leaving [232.6mb] used, max [4.1gb] > [2011-07-01 14:26:09,403][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][127] took [3ms]/[1.7s], reclaimed [33mb], leaving > [232.9mb] used, max [4.1gb] > [2011-07-01 14:26:12,451][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][128] took [3ms]/[1.7s], reclaimed [33.3mb], > leaving [232.8mb] used, max [4.1gb] > [2011-07-01 14:26:14,454][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][129] took [3ms]/[1.7s], reclaimed [33.4mb], > leaving [232.7mb] used, max [4.1gb] > [2011-07-01 14:26:17,461][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][130] took [3ms]/[1.7s], reclaimed [33.1mb], > leaving [232.9mb] used, max [4.1gb] > [2011-07-01 14:26:20,491][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][131] took [4ms]/[1.7s], reclaimed [33.4mb], > leaving [232.7mb] used, max [4.1gb] > [2011-07-01 14:26:24,544][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][132] took [4ms]/[1.7s], reclaimed [33.1mb], > leaving [232.9mb] used, max [4.1gb] > [2011-07-01 14:26:26,547][DEBUG][monitor.jvm] [Jenkins, > Abner] [gc][ParNew][133] took [3ms]/[1.7s], reclaimed [33.2mb], > leaving [232.9mb] used, max [4.1gb] > > This is elasticsearch on a single host. No fancy setup, pretty much > straight out of the box, but with 4 GB of memory. > > Dan > > On Jul 1, 12:55 pm, Shay Banon \<[shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) ([http://elasticsearch.com](http://elasticsearch.com))\> wrote: > > > It failed on one shard, anything in the logs? > > > > On Friday, July 1, 2011 at 7:49 PM, Dan Diephouse wrote: > > > > > I'm trying to execute a query to clean out some stuff from our > > > database, but it's failing. This same query works on another ES > > > instance that we have but without so much data in it. Any idea why I'm > > > getting this failure or where I look for more information? > > > > > curl -XDELETE '[http://host:9200/ion/dantest/\_query](http://host:9200/ion/dantest/_query)? > > > [q=fields.domain=dantest.muleion.com](http://q=fields.domain=dantest.muleion.com) ([http://dantest.muleion.com](http://dantest.muleion.com))' > > > {"ok":true,"\_indices":{"ion":{"\_shards":{"total":5,"successful": > > > 4,"failed":1}}}} > > > > > Cheers, > > > Dan ---
**Author:** ![Dan\_Diephouse](https://avatars.discourse-cdn.com/v4/letter/d/779978/32.png) [@Dan\_Diephouse](https://discuss.elastic.co/u/Dan_Diephouse)\ **Post date:** [July 3, 2011, 3:39pm UTC](https://discuss.elastic.co/t/cant-execute-bulk-query/4750/5 "2011-07-03T15:39:38Z")
Well, after I wrote my last message I tried this: curl -XDELETE [http://host:9200/ion/dantest](http://host:9200/ion/dantest) This succeeded. However, while it decreased my number of documents from 8.5 million to 1.5, my database stayed the exact same size (8 gb)! Any ideas there? I just tried my old query again and it still fails. I'm not sure I can upload the DB somewhere as it has customer sensitive data in it. Dan On Jul 2, 4:44 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > String..., when theres a failure, then it should be logged. The problem with the delete\_by\_query API is that it does not return a message of the failure (as opposed to other "broadcast" APIs, like search) ... . Anything I can try and recreate? Don't mind getting hte data dir if its test data, and I can try and repro it locally. > > On Friday, July 1, 2011 at 9:27 PM, Dan Diephouse wrote: > > > Command: > > > # date && curl -XDELETE '[http://host:9200/ion/dantest/\_query](http://host:9200/ion/dantest/_query)? > > > > [q=fields.domain=dantest.muleion.com](http://q=fields.domain=dantest.muleion.com) ([http://dantest.muleion.com](http://dantest.muleion.com))' > > && echo && date > > Fri Jul 1 14:25:32 EDT 2011 > > {"ok":true,"\_indices":{"ion":{"\_shards":{"total":5,"successful": > > 4,"failed":1}}}} > > Fri Jul 1 14:26:35 EDT 2011 > > > Logs: > > > [2011-07-01 14:25:32,792][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][103] took [6ms]/[1.4s], reclaimed [32.6mb], > > leaving [223.2mb] used, max [4.1gb] > > [2011-07-01 14:25:33,794][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][104] took [7ms]/[1.4s], reclaimed [33.1mb], > > leaving [223.4mb] used, max [4.1gb] > > [2011-07-01 14:25:34,794][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][105] took [6ms]/[1.4s], reclaimed [32.4mb], > > leaving [224.3mb] used, max [4.1gb] > > [2011-07-01 14:25:35,819][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][106] took [6ms]/[1.4s], reclaimed [33mb], leaving > > [224.6mb] used, max [4.1gb] > > [2011-07-01 14:25:36,821][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][108] took [9ms]/[1.4s], reclaimed [32.6mb], > > leaving [226.2mb] used, max [4.1gb] > > [2011-07-01 14:25:38,101][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][110] took [11ms]/[1.5s], reclaimed [32.2mb], > > leaving [227.5mb] used, max [4.1gb] > > [2011-07-01 14:25:39,103][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][112] took [7ms]/[1.5s], reclaimed [32.9mb], > > leaving [228.3mb] used, max [4.1gb] > > [2011-07-01 14:25:40,105][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][113] took [5ms]/[1.5s], reclaimed [32.9mb], > > leaving [228.6mb] used, max [4.1gb] > > [2011-07-01 14:25:41,107][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][114] took [6ms]/[1.5s], reclaimed [33.3mb], > > leaving [228.5mb] used, max [4.1gb] > > [2011-07-01 14:25:43,111][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][116] took [6ms]/[1.6s], reclaimed [32.7mb], > > leaving [229.9mb] used, max [4.1gb] > > [2011-07-01 14:25:46,171][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][117] took [6ms]/[1.6s], reclaimed [32.6mb], > > leaving [230.5mb] used, max [4.1gb] > > [2011-07-01 14:25:47,173][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][118] took [6ms]/[1.6s], reclaimed [32.8mb], > > leaving [231mb] used, max [4.1gb] > > [2011-07-01 14:25:48,175][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][119] took [21ms]/[1.6s], reclaimed [32.9mb], > > leaving [231.4mb] used, max [4.1gb] > > [2011-07-01 14:25:50,209][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][120] took [65ms]/[1.7s], reclaimed [33mb], leaving > > [231.6mb] used, max [4.1gb] > > [2011-07-01 14:25:51,211][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][121] took [8ms]/[1.7s], reclaimed [32.4mb], > > leaving [232.5mb] used, max [4.1gb] > > [2011-07-01 14:25:54,216][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][122] took [9ms]/[1.7s], reclaimed [33.3mb], > > leaving [232.4mb] used, max [4.1gb] > > [2011-07-01 14:25:57,219][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][123] took [7ms]/[1.7s], reclaimed [33mb], leaving > > [232.7mb] used, max [4.1gb] > > [2011-07-01 14:26:00,224][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][124] took [3ms]/[1.7s], reclaimed [33.2mb], > > leaving [232.8mb] used, max [4.1gb] > > [2011-07-01 14:26:02,280][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][125] took [4ms]/[1.7s], reclaimed [33.4mb], > > leaving [232.6mb] used, max [4.1gb] > > [2011-07-01 14:26:05,324][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][126] took [4ms]/[1.7s], reclaimed [33.3mb], > > leaving [232.6mb] used, max [4.1gb] > > [2011-07-01 14:26:09,403][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][127] took [3ms]/[1.7s], reclaimed [33mb], leaving > > [232.9mb] used, max [4.1gb] > > [2011-07-01 14:26:12,451][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][128] took [3ms]/[1.7s], reclaimed [33.3mb], > > leaving [232.8mb] used, max [4.1gb] > > [2011-07-01 14:26:14,454][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][129] took [3ms]/[1.7s], reclaimed [33.4mb], > > leaving [232.7mb] used, max [4.1gb] > > [2011-07-01 14:26:17,461][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][130] took [3ms]/[1.7s], reclaimed [33.1mb], > > leaving [232.9mb] used, max [4.1gb] > > [2011-07-01 14:26:20,491][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][131] took [4ms]/[1.7s], reclaimed [33.4mb], > > leaving [232.7mb] used, max [4.1gb] > > [2011-07-01 14:26:24,544][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][132] took [4ms]/[1.7s], reclaimed [33.1mb], > > leaving [232.9mb] used, max [4.1gb] > > [2011-07-01 14:26:26,547][DEBUG][monitor.jvm] [Jenkins, > > Abner] [gc][ParNew][133] took [3ms]/[1.7s], reclaimed [33.2mb], > > leaving [232.9mb] used, max [4.1gb] > > > This is elasticsearch on a single host. No fancy setup, pretty much > > straight out of the box, but with 4 GB of memory. > > > Dan > > > On Jul 1, 12:55 pm, Shay Banon \<[shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) ([http://elasticsearch.com](http://elasticsearch.com))\> wrote: > > > > > It failed on one shard, anything in the logs? > > > > On Friday, July 1, 2011 at 7:49 PM, Dan Diephouse wrote: > > > > > > > I'm trying to execute a query to clean out some stuff from our > > > > database, but it's failing. This same query works on another ES > > > > instance that we have but without so much data in it. Any idea why I'm > > > > getting this failure or where I look for more information? > > > > > curl -XDELETE '[http://host:9200/ion/dantest/\_query](http://host:9200/ion/dantest/_query)? > > > > [q=fields.domain=dantest.muleion.com](http://q=fields.domain=dantest.muleion.com) ([http://dantest.muleion.com](http://dantest.muleion.com))' > > > > {"ok":true,"\_indices":{"ion":{"\_shards":{"total":5,"successful": > > > > 4,"failed":1}}}} > > > > > Cheers, > > > > Dan ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [July 3, 2011, 3:42pm UTC](https://discuss.elastic.co/t/cant-execute-bulk-query/4750/6 "2011-07-03T15:42:26Z")
On Sun, 2011-07-03 at 08:39 -0700, Dan Diephouse wrote: > Well, after I wrote my last message I tried this: > > curl -XDELETE [http://host:9200/ion/dantest](http://host:9200/ion/dantest) > > This succeeded. However, while it decreased my number of documents > from 8.5 million to 1.5, my database stayed the exact same size (8 > gb)! Any ideas there? > > I just tried my old query again and it still fails. I'm not sure I > can upload the DB somewhere as it has customer sensitive data in it. When you delete docs, they are actually just marked as deleted, and are no longer returned by any queries. When ES merges segments (the data files that actually store your docs) then the deleted documents will be removed. You can force this using the optimize API: > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. Or you can reindex to a new index, which won't include the old deleted docs clint ---
**Author:** ![Dan\_Diephouse](https://avatars.discourse-cdn.com/v4/letter/d/779978/32.png) [@Dan\_Diephouse](https://discuss.elastic.co/u/Dan_Diephouse)\ **Post date:** [July 4, 2011, 5:44pm UTC](https://discuss.elastic.co/t/cant-execute-bulk-query/4750/7 "2011-07-04T17:44:41Z")
On Jul 3, 11:42 am, Clinton Gormley [clin...@iannounce.co.uk](mailto:clin...@iannounce.co.uk) wrote: > When you delete docs, they are actually just marked as deleted, and are > no longer returned by any queries. > > When ES merges segments (the data files that actually store your docs) > then the deleted documents will be removed. > > You can force this using the optimize API:[Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-optimi)... > > Or you can reindex to a new index, which won't include the old deleted > docs Thanks. Once I did this: curl -XPOST '[http://search.muleion.com:9200/ion/\_optimize](http://search.muleion.com:9200/ion/_optimize)? only\_expunge\_deletes=true' My index went down to the proper size. Dan ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:01am UTC](https://discuss.elastic.co/t/cant-execute-bulk-query/4750/8 "2017-07-06T04:01:51Z")