Can't execute bulk query


(Dan Diephouse) #1

I'm trying to execute a query to clean out some stuff from our
database, but it's failing. This same query works on another ES
instance that we have but without so much data in it. Any idea why I'm
getting this failure or where I look for more information?

curl -XDELETE 'http://host:9200/ion/dantest/_query?
q=fields.domain=dantest.muleion.com'
{"ok":true,"_indices":{"ion":{"_shards":{"total":5,"successful":
4,"failed":1}}}}

Cheers,
Dan


(Shay Banon) #2

It failed on one shard, anything in the logs?

On Friday, July 1, 2011 at 7:49 PM, Dan Diephouse wrote:

I'm trying to execute a query to clean out some stuff from our
database, but it's failing. This same query works on another ES
instance that we have but without so much data in it. Any idea why I'm
getting this failure or where I look for more information?

curl -XDELETE 'http://host:9200/ion/dantest/_query?
q=fields.domain=dantest.muleion.com (http://dantest.muleion.com)'
{"ok":true,"_indices":{"ion":{"_shards":{"total":5,"successful":
4,"failed":1}}}}

Cheers,
Dan


(Dan Diephouse) #3

Command:

date && curl -XDELETE 'http://host:9200/ion/dantest/_query?

q=fields.domain=dantest.muleion.com'
&& echo && date
Fri Jul 1 14:25:32 EDT 2011
{"ok":true,"_indices":{"ion":{"_shards":{"total":5,"successful":
4,"failed":1}}}}
Fri Jul 1 14:26:35 EDT 2011

Logs:

[2011-07-01 14:25:32,792][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][103] took [6ms]/[1.4s], reclaimed [32.6mb],
leaving [223.2mb] used, max [4.1gb]
[2011-07-01 14:25:33,794][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][104] took [7ms]/[1.4s], reclaimed [33.1mb],
leaving [223.4mb] used, max [4.1gb]
[2011-07-01 14:25:34,794][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][105] took [6ms]/[1.4s], reclaimed [32.4mb],
leaving [224.3mb] used, max [4.1gb]
[2011-07-01 14:25:35,819][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][106] took [6ms]/[1.4s], reclaimed [33mb], leaving
[224.6mb] used, max [4.1gb]
[2011-07-01 14:25:36,821][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][108] took [9ms]/[1.4s], reclaimed [32.6mb],
leaving [226.2mb] used, max [4.1gb]
[2011-07-01 14:25:38,101][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][110] took [11ms]/[1.5s], reclaimed [32.2mb],
leaving [227.5mb] used, max [4.1gb]
[2011-07-01 14:25:39,103][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][112] took [7ms]/[1.5s], reclaimed [32.9mb],
leaving [228.3mb] used, max [4.1gb]
[2011-07-01 14:25:40,105][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][113] took [5ms]/[1.5s], reclaimed [32.9mb],
leaving [228.6mb] used, max [4.1gb]
[2011-07-01 14:25:41,107][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][114] took [6ms]/[1.5s], reclaimed [33.3mb],
leaving [228.5mb] used, max [4.1gb]
[2011-07-01 14:25:43,111][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][116] took [6ms]/[1.6s], reclaimed [32.7mb],
leaving [229.9mb] used, max [4.1gb]
[2011-07-01 14:25:46,171][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][117] took [6ms]/[1.6s], reclaimed [32.6mb],
leaving [230.5mb] used, max [4.1gb]
[2011-07-01 14:25:47,173][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][118] took [6ms]/[1.6s], reclaimed [32.8mb],
leaving [231mb] used, max [4.1gb]
[2011-07-01 14:25:48,175][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][119] took [21ms]/[1.6s], reclaimed [32.9mb],
leaving [231.4mb] used, max [4.1gb]
[2011-07-01 14:25:50,209][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][120] took [65ms]/[1.7s], reclaimed [33mb], leaving
[231.6mb] used, max [4.1gb]
[2011-07-01 14:25:51,211][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][121] took [8ms]/[1.7s], reclaimed [32.4mb],
leaving [232.5mb] used, max [4.1gb]
[2011-07-01 14:25:54,216][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][122] took [9ms]/[1.7s], reclaimed [33.3mb],
leaving [232.4mb] used, max [4.1gb]
[2011-07-01 14:25:57,219][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][123] took [7ms]/[1.7s], reclaimed [33mb], leaving
[232.7mb] used, max [4.1gb]
[2011-07-01 14:26:00,224][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][124] took [3ms]/[1.7s], reclaimed [33.2mb],
leaving [232.8mb] used, max [4.1gb]
[2011-07-01 14:26:02,280][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][125] took [4ms]/[1.7s], reclaimed [33.4mb],
leaving [232.6mb] used, max [4.1gb]
[2011-07-01 14:26:05,324][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][126] took [4ms]/[1.7s], reclaimed [33.3mb],
leaving [232.6mb] used, max [4.1gb]
[2011-07-01 14:26:09,403][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][127] took [3ms]/[1.7s], reclaimed [33mb], leaving
[232.9mb] used, max [4.1gb]
[2011-07-01 14:26:12,451][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][128] took [3ms]/[1.7s], reclaimed [33.3mb],
leaving [232.8mb] used, max [4.1gb]
[2011-07-01 14:26:14,454][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][129] took [3ms]/[1.7s], reclaimed [33.4mb],
leaving [232.7mb] used, max [4.1gb]
[2011-07-01 14:26:17,461][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][130] took [3ms]/[1.7s], reclaimed [33.1mb],
leaving [232.9mb] used, max [4.1gb]
[2011-07-01 14:26:20,491][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][131] took [4ms]/[1.7s], reclaimed [33.4mb],
leaving [232.7mb] used, max [4.1gb]
[2011-07-01 14:26:24,544][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][132] took [4ms]/[1.7s], reclaimed [33.1mb],
leaving [232.9mb] used, max [4.1gb]
[2011-07-01 14:26:26,547][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][133] took [3ms]/[1.7s], reclaimed [33.2mb],
leaving [232.9mb] used, max [4.1gb]

This is elasticsearch on a single host. No fancy setup, pretty much
straight out of the box, but with 4 GB of memory.

Dan

On Jul 1, 12:55 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

It failed on one shard, anything in the logs?

On Friday, July 1, 2011 at 7:49 PM, Dan Diephouse wrote:

I'm trying to execute a query to clean out some stuff from our
database, but it's failing. This same query works on another ES
instance that we have but without so much data in it. Any idea why I'm
getting this failure or where I look for more information?

curl -XDELETE 'http://host:9200/ion/dantest/_query?
q=fields.domain=dantest.muleion.com (http://dantest.muleion.com)'
{"ok":true,"_indices":{"ion":{"_shards":{"total":5,"successful":
4,"failed":1}}}}

Cheers,
Dan


(Shay Banon) #4

String..., when theres a failure, then it should be logged. The problem with the delete_by_query API is that it does not return a message of the failure (as opposed to other "broadcast" APIs, like search) ... . Anything I can try and recreate? Don't mind getting hte data dir if its test data, and I can try and repro it locally.

On Friday, July 1, 2011 at 9:27 PM, Dan Diephouse wrote:

Command:

date && curl -XDELETE 'http://host:9200/ion/dantest/_query?

q=fields.domain=dantest.muleion.com (http://dantest.muleion.com)'
&& echo && date
Fri Jul 1 14:25:32 EDT 2011
{"ok":true,"_indices":{"ion":{"_shards":{"total":5,"successful":
4,"failed":1}}}}
Fri Jul 1 14:26:35 EDT 2011

Logs:

[2011-07-01 14:25:32,792][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][103] took [6ms]/[1.4s], reclaimed [32.6mb],
leaving [223.2mb] used, max [4.1gb]
[2011-07-01 14:25:33,794][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][104] took [7ms]/[1.4s], reclaimed [33.1mb],
leaving [223.4mb] used, max [4.1gb]
[2011-07-01 14:25:34,794][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][105] took [6ms]/[1.4s], reclaimed [32.4mb],
leaving [224.3mb] used, max [4.1gb]
[2011-07-01 14:25:35,819][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][106] took [6ms]/[1.4s], reclaimed [33mb], leaving
[224.6mb] used, max [4.1gb]
[2011-07-01 14:25:36,821][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][108] took [9ms]/[1.4s], reclaimed [32.6mb],
leaving [226.2mb] used, max [4.1gb]
[2011-07-01 14:25:38,101][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][110] took [11ms]/[1.5s], reclaimed [32.2mb],
leaving [227.5mb] used, max [4.1gb]
[2011-07-01 14:25:39,103][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][112] took [7ms]/[1.5s], reclaimed [32.9mb],
leaving [228.3mb] used, max [4.1gb]
[2011-07-01 14:25:40,105][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][113] took [5ms]/[1.5s], reclaimed [32.9mb],
leaving [228.6mb] used, max [4.1gb]
[2011-07-01 14:25:41,107][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][114] took [6ms]/[1.5s], reclaimed [33.3mb],
leaving [228.5mb] used, max [4.1gb]
[2011-07-01 14:25:43,111][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][116] took [6ms]/[1.6s], reclaimed [32.7mb],
leaving [229.9mb] used, max [4.1gb]
[2011-07-01 14:25:46,171][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][117] took [6ms]/[1.6s], reclaimed [32.6mb],
leaving [230.5mb] used, max [4.1gb]
[2011-07-01 14:25:47,173][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][118] took [6ms]/[1.6s], reclaimed [32.8mb],
leaving [231mb] used, max [4.1gb]
[2011-07-01 14:25:48,175][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][119] took [21ms]/[1.6s], reclaimed [32.9mb],
leaving [231.4mb] used, max [4.1gb]
[2011-07-01 14:25:50,209][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][120] took [65ms]/[1.7s], reclaimed [33mb], leaving
[231.6mb] used, max [4.1gb]
[2011-07-01 14:25:51,211][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][121] took [8ms]/[1.7s], reclaimed [32.4mb],
leaving [232.5mb] used, max [4.1gb]
[2011-07-01 14:25:54,216][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][122] took [9ms]/[1.7s], reclaimed [33.3mb],
leaving [232.4mb] used, max [4.1gb]
[2011-07-01 14:25:57,219][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][123] took [7ms]/[1.7s], reclaimed [33mb], leaving
[232.7mb] used, max [4.1gb]
[2011-07-01 14:26:00,224][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][124] took [3ms]/[1.7s], reclaimed [33.2mb],
leaving [232.8mb] used, max [4.1gb]
[2011-07-01 14:26:02,280][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][125] took [4ms]/[1.7s], reclaimed [33.4mb],
leaving [232.6mb] used, max [4.1gb]
[2011-07-01 14:26:05,324][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][126] took [4ms]/[1.7s], reclaimed [33.3mb],
leaving [232.6mb] used, max [4.1gb]
[2011-07-01 14:26:09,403][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][127] took [3ms]/[1.7s], reclaimed [33mb], leaving
[232.9mb] used, max [4.1gb]
[2011-07-01 14:26:12,451][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][128] took [3ms]/[1.7s], reclaimed [33.3mb],
leaving [232.8mb] used, max [4.1gb]
[2011-07-01 14:26:14,454][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][129] took [3ms]/[1.7s], reclaimed [33.4mb],
leaving [232.7mb] used, max [4.1gb]
[2011-07-01 14:26:17,461][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][130] took [3ms]/[1.7s], reclaimed [33.1mb],
leaving [232.9mb] used, max [4.1gb]
[2011-07-01 14:26:20,491][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][131] took [4ms]/[1.7s], reclaimed [33.4mb],
leaving [232.7mb] used, max [4.1gb]
[2011-07-01 14:26:24,544][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][132] took [4ms]/[1.7s], reclaimed [33.1mb],
leaving [232.9mb] used, max [4.1gb]
[2011-07-01 14:26:26,547][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][133] took [3ms]/[1.7s], reclaimed [33.2mb],
leaving [232.9mb] used, max [4.1gb]

This is elasticsearch on a single host. No fancy setup, pretty much
straight out of the box, but with 4 GB of memory.

Dan

On Jul 1, 12:55 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

It failed on one shard, anything in the logs?

On Friday, July 1, 2011 at 7:49 PM, Dan Diephouse wrote:

I'm trying to execute a query to clean out some stuff from our
database, but it's failing. This same query works on another ES
instance that we have but without so much data in it. Any idea why I'm
getting this failure or where I look for more information?

curl -XDELETE 'http://host:9200/ion/dantest/_query?
q=fields.domain=dantest.muleion.com (http://dantest.muleion.com)'
{"ok":true,"_indices":{"ion":{"_shards":{"total":5,"successful":
4,"failed":1}}}}

Cheers,
Dan


(Dan Diephouse) #5

Well, after I wrote my last message I tried this:

curl -XDELETE http://host:9200/ion/dantest

This succeeded. However, while it decreased my number of documents
from 8.5 million to 1.5, my database stayed the exact same size (8
gb)! Any ideas there?

I just tried my old query again and it still fails. I'm not sure I
can upload the DB somewhere as it has customer sensitive data in it.

Dan

On Jul 2, 4:44 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

String..., when theres a failure, then it should be logged. The problem with the delete_by_query API is that it does not return a message of the failure (as opposed to other "broadcast" APIs, like search) ... . Anything I can try and recreate? Don't mind getting hte data dir if its test data, and I can try and repro it locally.

On Friday, July 1, 2011 at 9:27 PM, Dan Diephouse wrote:

Command:

date && curl -XDELETE 'http://host:9200/ion/dantest/_query?

q=fields.domain=dantest.muleion.com (http://dantest.muleion.com)'
&& echo && date
Fri Jul 1 14:25:32 EDT 2011
{"ok":true,"_indices":{"ion":{"_shards":{"total":5,"successful":
4,"failed":1}}}}
Fri Jul 1 14:26:35 EDT 2011

Logs:

[2011-07-01 14:25:32,792][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][103] took [6ms]/[1.4s], reclaimed [32.6mb],
leaving [223.2mb] used, max [4.1gb]
[2011-07-01 14:25:33,794][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][104] took [7ms]/[1.4s], reclaimed [33.1mb],
leaving [223.4mb] used, max [4.1gb]
[2011-07-01 14:25:34,794][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][105] took [6ms]/[1.4s], reclaimed [32.4mb],
leaving [224.3mb] used, max [4.1gb]
[2011-07-01 14:25:35,819][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][106] took [6ms]/[1.4s], reclaimed [33mb], leaving
[224.6mb] used, max [4.1gb]
[2011-07-01 14:25:36,821][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][108] took [9ms]/[1.4s], reclaimed [32.6mb],
leaving [226.2mb] used, max [4.1gb]
[2011-07-01 14:25:38,101][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][110] took [11ms]/[1.5s], reclaimed [32.2mb],
leaving [227.5mb] used, max [4.1gb]
[2011-07-01 14:25:39,103][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][112] took [7ms]/[1.5s], reclaimed [32.9mb],
leaving [228.3mb] used, max [4.1gb]
[2011-07-01 14:25:40,105][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][113] took [5ms]/[1.5s], reclaimed [32.9mb],
leaving [228.6mb] used, max [4.1gb]
[2011-07-01 14:25:41,107][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][114] took [6ms]/[1.5s], reclaimed [33.3mb],
leaving [228.5mb] used, max [4.1gb]
[2011-07-01 14:25:43,111][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][116] took [6ms]/[1.6s], reclaimed [32.7mb],
leaving [229.9mb] used, max [4.1gb]
[2011-07-01 14:25:46,171][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][117] took [6ms]/[1.6s], reclaimed [32.6mb],
leaving [230.5mb] used, max [4.1gb]
[2011-07-01 14:25:47,173][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][118] took [6ms]/[1.6s], reclaimed [32.8mb],
leaving [231mb] used, max [4.1gb]
[2011-07-01 14:25:48,175][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][119] took [21ms]/[1.6s], reclaimed [32.9mb],
leaving [231.4mb] used, max [4.1gb]
[2011-07-01 14:25:50,209][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][120] took [65ms]/[1.7s], reclaimed [33mb], leaving
[231.6mb] used, max [4.1gb]
[2011-07-01 14:25:51,211][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][121] took [8ms]/[1.7s], reclaimed [32.4mb],
leaving [232.5mb] used, max [4.1gb]
[2011-07-01 14:25:54,216][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][122] took [9ms]/[1.7s], reclaimed [33.3mb],
leaving [232.4mb] used, max [4.1gb]
[2011-07-01 14:25:57,219][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][123] took [7ms]/[1.7s], reclaimed [33mb], leaving
[232.7mb] used, max [4.1gb]
[2011-07-01 14:26:00,224][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][124] took [3ms]/[1.7s], reclaimed [33.2mb],
leaving [232.8mb] used, max [4.1gb]
[2011-07-01 14:26:02,280][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][125] took [4ms]/[1.7s], reclaimed [33.4mb],
leaving [232.6mb] used, max [4.1gb]
[2011-07-01 14:26:05,324][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][126] took [4ms]/[1.7s], reclaimed [33.3mb],
leaving [232.6mb] used, max [4.1gb]
[2011-07-01 14:26:09,403][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][127] took [3ms]/[1.7s], reclaimed [33mb], leaving
[232.9mb] used, max [4.1gb]
[2011-07-01 14:26:12,451][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][128] took [3ms]/[1.7s], reclaimed [33.3mb],
leaving [232.8mb] used, max [4.1gb]
[2011-07-01 14:26:14,454][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][129] took [3ms]/[1.7s], reclaimed [33.4mb],
leaving [232.7mb] used, max [4.1gb]
[2011-07-01 14:26:17,461][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][130] took [3ms]/[1.7s], reclaimed [33.1mb],
leaving [232.9mb] used, max [4.1gb]
[2011-07-01 14:26:20,491][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][131] took [4ms]/[1.7s], reclaimed [33.4mb],
leaving [232.7mb] used, max [4.1gb]
[2011-07-01 14:26:24,544][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][132] took [4ms]/[1.7s], reclaimed [33.1mb],
leaving [232.9mb] used, max [4.1gb]
[2011-07-01 14:26:26,547][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][133] took [3ms]/[1.7s], reclaimed [33.2mb],
leaving [232.9mb] used, max [4.1gb]

This is elasticsearch on a single host. No fancy setup, pretty much
straight out of the box, but with 4 GB of memory.

Dan

On Jul 1, 12:55 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:

It failed on one shard, anything in the logs?

On Friday, July 1, 2011 at 7:49 PM, Dan Diephouse wrote:

I'm trying to execute a query to clean out some stuff from our
database, but it's failing. This same query works on another ES
instance that we have but without so much data in it. Any idea why I'm
getting this failure or where I look for more information?

curl -XDELETE 'http://host:9200/ion/dantest/_query?
q=fields.domain=dantest.muleion.com (http://dantest.muleion.com)'
{"ok":true,"_indices":{"ion":{"_shards":{"total":5,"successful":
4,"failed":1}}}}

Cheers,
Dan


(Clinton Gormley) #6

On Sun, 2011-07-03 at 08:39 -0700, Dan Diephouse wrote:

Well, after I wrote my last message I tried this:

curl -XDELETE http://host:9200/ion/dantest

This succeeded. However, while it decreased my number of documents
from 8.5 million to 1.5, my database stayed the exact same size (8
gb)! Any ideas there?

I just tried my old query again and it still fails. I'm not sure I
can upload the DB somewhere as it has customer sensitive data in it.

When you delete docs, they are actually just marked as deleted, and are
no longer returned by any queries.

When ES merges segments (the data files that actually store your docs)
then the deleted documents will be removed.

You can force this using the optimize API:
http://www.elasticsearch.org/guide/reference/api/admin-indices-optimize.html

Or you can reindex to a new index, which won't include the old deleted
docs

clint


(Dan Diephouse) #7

On Jul 3, 11:42 am, Clinton Gormley clin...@iannounce.co.uk wrote:

When you delete docs, they are actually just marked as deleted, and are
no longer returned by any queries.

When ES merges segments (the data files that actually store your docs)
then the deleted documents will be removed.

You can force this using the optimize API:http://www.elasticsearch.org/guide/reference/api/admin-indices-optimi...

Or you can reindex to a new index, which won't include the old deleted
docs

Thanks. Once I did this:

curl -XPOST 'http://search.muleion.com:9200/ion/_optimize?
only_expunge_deletes=true'

My index went down to the proper size.

Dan


(system) #8