I'm trying to execute a query to clean out some stuff from our
database, but it's failing. This same query works on another ES
instance that we have but without so much data in it. Any idea why I'm
getting this failure or where I look for more information?
curl -XDELETE 'http://host:9200/ion/dantest/_query?
q=fields.domain=dantest.muleion.com'
{"ok":true,"_indices":{"ion":{"_shards":{"total":5,"successful":
4,"failed":1}}}}
Cheers,
Dan
It failed on one shard, anything in the logs?
On Friday, July 1, 2011 at 7:49 PM, Dan Diephouse wrote:
I'm trying to execute a query to clean out some stuff from our
database, but it's failing. This same query works on another ES
instance that we have but without so much data in it. Any idea why I'm
getting this failure or where I look for more information?curl -XDELETE 'http://host:9200/ion/dantest/_query?
q=fields.domain=dantest.muleion.com (http://dantest.muleion.com)'
{"ok":true,"_indices":{"ion":{"_shards":{"total":5,"successful":
4,"failed":1}}}}Cheers,
Dan
Command:
q=fields.domain=dantest.muleion.com'
&& echo && date
Fri Jul 1 14:25:32 EDT 2011
{"ok":true,"_indices":{"ion":{"_shards":{"total":5,"successful":
4,"failed":1}}}}
Fri Jul 1 14:26:35 EDT 2011
Logs:
[2011-07-01 14:25:32,792][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][103] took [6ms]/[1.4s], reclaimed [32.6mb],
leaving [223.2mb] used, max [4.1gb]
[2011-07-01 14:25:33,794][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][104] took [7ms]/[1.4s], reclaimed [33.1mb],
leaving [223.4mb] used, max [4.1gb]
[2011-07-01 14:25:34,794][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][105] took [6ms]/[1.4s], reclaimed [32.4mb],
leaving [224.3mb] used, max [4.1gb]
[2011-07-01 14:25:35,819][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][106] took [6ms]/[1.4s], reclaimed [33mb], leaving
[224.6mb] used, max [4.1gb]
[2011-07-01 14:25:36,821][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][108] took [9ms]/[1.4s], reclaimed [32.6mb],
leaving [226.2mb] used, max [4.1gb]
[2011-07-01 14:25:38,101][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][110] took [11ms]/[1.5s], reclaimed [32.2mb],
leaving [227.5mb] used, max [4.1gb]
[2011-07-01 14:25:39,103][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][112] took [7ms]/[1.5s], reclaimed [32.9mb],
leaving [228.3mb] used, max [4.1gb]
[2011-07-01 14:25:40,105][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][113] took [5ms]/[1.5s], reclaimed [32.9mb],
leaving [228.6mb] used, max [4.1gb]
[2011-07-01 14:25:41,107][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][114] took [6ms]/[1.5s], reclaimed [33.3mb],
leaving [228.5mb] used, max [4.1gb]
[2011-07-01 14:25:43,111][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][116] took [6ms]/[1.6s], reclaimed [32.7mb],
leaving [229.9mb] used, max [4.1gb]
[2011-07-01 14:25:46,171][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][117] took [6ms]/[1.6s], reclaimed [32.6mb],
leaving [230.5mb] used, max [4.1gb]
[2011-07-01 14:25:47,173][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][118] took [6ms]/[1.6s], reclaimed [32.8mb],
leaving [231mb] used, max [4.1gb]
[2011-07-01 14:25:48,175][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][119] took [21ms]/[1.6s], reclaimed [32.9mb],
leaving [231.4mb] used, max [4.1gb]
[2011-07-01 14:25:50,209][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][120] took [65ms]/[1.7s], reclaimed [33mb], leaving
[231.6mb] used, max [4.1gb]
[2011-07-01 14:25:51,211][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][121] took [8ms]/[1.7s], reclaimed [32.4mb],
leaving [232.5mb] used, max [4.1gb]
[2011-07-01 14:25:54,216][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][122] took [9ms]/[1.7s], reclaimed [33.3mb],
leaving [232.4mb] used, max [4.1gb]
[2011-07-01 14:25:57,219][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][123] took [7ms]/[1.7s], reclaimed [33mb], leaving
[232.7mb] used, max [4.1gb]
[2011-07-01 14:26:00,224][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][124] took [3ms]/[1.7s], reclaimed [33.2mb],
leaving [232.8mb] used, max [4.1gb]
[2011-07-01 14:26:02,280][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][125] took [4ms]/[1.7s], reclaimed [33.4mb],
leaving [232.6mb] used, max [4.1gb]
[2011-07-01 14:26:05,324][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][126] took [4ms]/[1.7s], reclaimed [33.3mb],
leaving [232.6mb] used, max [4.1gb]
[2011-07-01 14:26:09,403][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][127] took [3ms]/[1.7s], reclaimed [33mb], leaving
[232.9mb] used, max [4.1gb]
[2011-07-01 14:26:12,451][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][128] took [3ms]/[1.7s], reclaimed [33.3mb],
leaving [232.8mb] used, max [4.1gb]
[2011-07-01 14:26:14,454][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][129] took [3ms]/[1.7s], reclaimed [33.4mb],
leaving [232.7mb] used, max [4.1gb]
[2011-07-01 14:26:17,461][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][130] took [3ms]/[1.7s], reclaimed [33.1mb],
leaving [232.9mb] used, max [4.1gb]
[2011-07-01 14:26:20,491][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][131] took [4ms]/[1.7s], reclaimed [33.4mb],
leaving [232.7mb] used, max [4.1gb]
[2011-07-01 14:26:24,544][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][132] took [4ms]/[1.7s], reclaimed [33.1mb],
leaving [232.9mb] used, max [4.1gb]
[2011-07-01 14:26:26,547][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][133] took [3ms]/[1.7s], reclaimed [33.2mb],
leaving [232.9mb] used, max [4.1gb]
This is elasticsearch on a single host. No fancy setup, pretty much
straight out of the box, but with 4 GB of memory.
Dan
On Jul 1, 12:55 pm, Shay Banon shay.ba...@elasticsearch.com wrote:
It failed on one shard, anything in the logs?
On Friday, July 1, 2011 at 7:49 PM, Dan Diephouse wrote:
I'm trying to execute a query to clean out some stuff from our
database, but it's failing. This same query works on another ES
instance that we have but without so much data in it. Any idea why I'm
getting this failure or where I look for more information?curl -XDELETE 'http://host:9200/ion/dantest/_query?
q=fields.domain=dantest.muleion.com (http://dantest.muleion.com)'
{"ok":true,"_indices":{"ion":{"_shards":{"total":5,"successful":
4,"failed":1}}}}Cheers,
Dan
String..., when theres a failure, then it should be logged. The problem with the delete_by_query API is that it does not return a message of the failure (as opposed to other "broadcast" APIs, like search) ... . Anything I can try and recreate? Don't mind getting hte data dir if its test data, and I can try and repro it locally.
On Friday, July 1, 2011 at 9:27 PM, Dan Diephouse wrote:
Command:
date && curl -XDELETE 'http://host:9200/ion/dantest/_query?
q=fields.domain=dantest.muleion.com (http://dantest.muleion.com)'
&& echo && date
Fri Jul 1 14:25:32 EDT 2011
{"ok":true,"_indices":{"ion":{"_shards":{"total":5,"successful":
4,"failed":1}}}}
Fri Jul 1 14:26:35 EDT 2011Logs:
[2011-07-01 14:25:32,792][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][103] took [6ms]/[1.4s], reclaimed [32.6mb],
leaving [223.2mb] used, max [4.1gb]
[2011-07-01 14:25:33,794][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][104] took [7ms]/[1.4s], reclaimed [33.1mb],
leaving [223.4mb] used, max [4.1gb]
[2011-07-01 14:25:34,794][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][105] took [6ms]/[1.4s], reclaimed [32.4mb],
leaving [224.3mb] used, max [4.1gb]
[2011-07-01 14:25:35,819][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][106] took [6ms]/[1.4s], reclaimed [33mb], leaving
[224.6mb] used, max [4.1gb]
[2011-07-01 14:25:36,821][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][108] took [9ms]/[1.4s], reclaimed [32.6mb],
leaving [226.2mb] used, max [4.1gb]
[2011-07-01 14:25:38,101][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][110] took [11ms]/[1.5s], reclaimed [32.2mb],
leaving [227.5mb] used, max [4.1gb]
[2011-07-01 14:25:39,103][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][112] took [7ms]/[1.5s], reclaimed [32.9mb],
leaving [228.3mb] used, max [4.1gb]
[2011-07-01 14:25:40,105][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][113] took [5ms]/[1.5s], reclaimed [32.9mb],
leaving [228.6mb] used, max [4.1gb]
[2011-07-01 14:25:41,107][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][114] took [6ms]/[1.5s], reclaimed [33.3mb],
leaving [228.5mb] used, max [4.1gb]
[2011-07-01 14:25:43,111][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][116] took [6ms]/[1.6s], reclaimed [32.7mb],
leaving [229.9mb] used, max [4.1gb]
[2011-07-01 14:25:46,171][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][117] took [6ms]/[1.6s], reclaimed [32.6mb],
leaving [230.5mb] used, max [4.1gb]
[2011-07-01 14:25:47,173][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][118] took [6ms]/[1.6s], reclaimed [32.8mb],
leaving [231mb] used, max [4.1gb]
[2011-07-01 14:25:48,175][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][119] took [21ms]/[1.6s], reclaimed [32.9mb],
leaving [231.4mb] used, max [4.1gb]
[2011-07-01 14:25:50,209][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][120] took [65ms]/[1.7s], reclaimed [33mb], leaving
[231.6mb] used, max [4.1gb]
[2011-07-01 14:25:51,211][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][121] took [8ms]/[1.7s], reclaimed [32.4mb],
leaving [232.5mb] used, max [4.1gb]
[2011-07-01 14:25:54,216][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][122] took [9ms]/[1.7s], reclaimed [33.3mb],
leaving [232.4mb] used, max [4.1gb]
[2011-07-01 14:25:57,219][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][123] took [7ms]/[1.7s], reclaimed [33mb], leaving
[232.7mb] used, max [4.1gb]
[2011-07-01 14:26:00,224][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][124] took [3ms]/[1.7s], reclaimed [33.2mb],
leaving [232.8mb] used, max [4.1gb]
[2011-07-01 14:26:02,280][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][125] took [4ms]/[1.7s], reclaimed [33.4mb],
leaving [232.6mb] used, max [4.1gb]
[2011-07-01 14:26:05,324][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][126] took [4ms]/[1.7s], reclaimed [33.3mb],
leaving [232.6mb] used, max [4.1gb]
[2011-07-01 14:26:09,403][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][127] took [3ms]/[1.7s], reclaimed [33mb], leaving
[232.9mb] used, max [4.1gb]
[2011-07-01 14:26:12,451][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][128] took [3ms]/[1.7s], reclaimed [33.3mb],
leaving [232.8mb] used, max [4.1gb]
[2011-07-01 14:26:14,454][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][129] took [3ms]/[1.7s], reclaimed [33.4mb],
leaving [232.7mb] used, max [4.1gb]
[2011-07-01 14:26:17,461][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][130] took [3ms]/[1.7s], reclaimed [33.1mb],
leaving [232.9mb] used, max [4.1gb]
[2011-07-01 14:26:20,491][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][131] took [4ms]/[1.7s], reclaimed [33.4mb],
leaving [232.7mb] used, max [4.1gb]
[2011-07-01 14:26:24,544][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][132] took [4ms]/[1.7s], reclaimed [33.1mb],
leaving [232.9mb] used, max [4.1gb]
[2011-07-01 14:26:26,547][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][133] took [3ms]/[1.7s], reclaimed [33.2mb],
leaving [232.9mb] used, max [4.1gb]This is elasticsearch on a single host. No fancy setup, pretty much
straight out of the box, but with 4 GB of memory.Dan
On Jul 1, 12:55 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:
It failed on one shard, anything in the logs?
On Friday, July 1, 2011 at 7:49 PM, Dan Diephouse wrote:
I'm trying to execute a query to clean out some stuff from our
database, but it's failing. This same query works on another ES
instance that we have but without so much data in it. Any idea why I'm
getting this failure or where I look for more information?curl -XDELETE 'http://host:9200/ion/dantest/_query?
q=fields.domain=dantest.muleion.com (http://dantest.muleion.com)'
{"ok":true,"_indices":{"ion":{"_shards":{"total":5,"successful":
4,"failed":1}}}}Cheers,
Dan
Well, after I wrote my last message I tried this:
curl -XDELETE http://host:9200/ion/dantest
This succeeded. However, while it decreased my number of documents
from 8.5 million to 1.5, my database stayed the exact same size (8
gb)! Any ideas there?
I just tried my old query again and it still fails. I'm not sure I
can upload the DB somewhere as it has customer sensitive data in it.
Dan
On Jul 2, 4:44 pm, Shay Banon shay.ba...@elasticsearch.com wrote:
String..., when theres a failure, then it should be logged. The problem with the delete_by_query API is that it does not return a message of the failure (as opposed to other "broadcast" APIs, like search) ... . Anything I can try and recreate? Don't mind getting hte data dir if its test data, and I can try and repro it locally.
On Friday, July 1, 2011 at 9:27 PM, Dan Diephouse wrote:
Command:
date && curl -XDELETE 'http://host:9200/ion/dantest/_query?
q=fields.domain=dantest.muleion.com (http://dantest.muleion.com)'
&& echo && date
Fri Jul 1 14:25:32 EDT 2011
{"ok":true,"_indices":{"ion":{"_shards":{"total":5,"successful":
4,"failed":1}}}}
Fri Jul 1 14:26:35 EDT 2011Logs:
[2011-07-01 14:25:32,792][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][103] took [6ms]/[1.4s], reclaimed [32.6mb],
leaving [223.2mb] used, max [4.1gb]
[2011-07-01 14:25:33,794][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][104] took [7ms]/[1.4s], reclaimed [33.1mb],
leaving [223.4mb] used, max [4.1gb]
[2011-07-01 14:25:34,794][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][105] took [6ms]/[1.4s], reclaimed [32.4mb],
leaving [224.3mb] used, max [4.1gb]
[2011-07-01 14:25:35,819][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][106] took [6ms]/[1.4s], reclaimed [33mb], leaving
[224.6mb] used, max [4.1gb]
[2011-07-01 14:25:36,821][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][108] took [9ms]/[1.4s], reclaimed [32.6mb],
leaving [226.2mb] used, max [4.1gb]
[2011-07-01 14:25:38,101][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][110] took [11ms]/[1.5s], reclaimed [32.2mb],
leaving [227.5mb] used, max [4.1gb]
[2011-07-01 14:25:39,103][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][112] took [7ms]/[1.5s], reclaimed [32.9mb],
leaving [228.3mb] used, max [4.1gb]
[2011-07-01 14:25:40,105][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][113] took [5ms]/[1.5s], reclaimed [32.9mb],
leaving [228.6mb] used, max [4.1gb]
[2011-07-01 14:25:41,107][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][114] took [6ms]/[1.5s], reclaimed [33.3mb],
leaving [228.5mb] used, max [4.1gb]
[2011-07-01 14:25:43,111][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][116] took [6ms]/[1.6s], reclaimed [32.7mb],
leaving [229.9mb] used, max [4.1gb]
[2011-07-01 14:25:46,171][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][117] took [6ms]/[1.6s], reclaimed [32.6mb],
leaving [230.5mb] used, max [4.1gb]
[2011-07-01 14:25:47,173][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][118] took [6ms]/[1.6s], reclaimed [32.8mb],
leaving [231mb] used, max [4.1gb]
[2011-07-01 14:25:48,175][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][119] took [21ms]/[1.6s], reclaimed [32.9mb],
leaving [231.4mb] used, max [4.1gb]
[2011-07-01 14:25:50,209][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][120] took [65ms]/[1.7s], reclaimed [33mb], leaving
[231.6mb] used, max [4.1gb]
[2011-07-01 14:25:51,211][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][121] took [8ms]/[1.7s], reclaimed [32.4mb],
leaving [232.5mb] used, max [4.1gb]
[2011-07-01 14:25:54,216][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][122] took [9ms]/[1.7s], reclaimed [33.3mb],
leaving [232.4mb] used, max [4.1gb]
[2011-07-01 14:25:57,219][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][123] took [7ms]/[1.7s], reclaimed [33mb], leaving
[232.7mb] used, max [4.1gb]
[2011-07-01 14:26:00,224][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][124] took [3ms]/[1.7s], reclaimed [33.2mb],
leaving [232.8mb] used, max [4.1gb]
[2011-07-01 14:26:02,280][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][125] took [4ms]/[1.7s], reclaimed [33.4mb],
leaving [232.6mb] used, max [4.1gb]
[2011-07-01 14:26:05,324][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][126] took [4ms]/[1.7s], reclaimed [33.3mb],
leaving [232.6mb] used, max [4.1gb]
[2011-07-01 14:26:09,403][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][127] took [3ms]/[1.7s], reclaimed [33mb], leaving
[232.9mb] used, max [4.1gb]
[2011-07-01 14:26:12,451][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][128] took [3ms]/[1.7s], reclaimed [33.3mb],
leaving [232.8mb] used, max [4.1gb]
[2011-07-01 14:26:14,454][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][129] took [3ms]/[1.7s], reclaimed [33.4mb],
leaving [232.7mb] used, max [4.1gb]
[2011-07-01 14:26:17,461][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][130] took [3ms]/[1.7s], reclaimed [33.1mb],
leaving [232.9mb] used, max [4.1gb]
[2011-07-01 14:26:20,491][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][131] took [4ms]/[1.7s], reclaimed [33.4mb],
leaving [232.7mb] used, max [4.1gb]
[2011-07-01 14:26:24,544][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][132] took [4ms]/[1.7s], reclaimed [33.1mb],
leaving [232.9mb] used, max [4.1gb]
[2011-07-01 14:26:26,547][DEBUG][monitor.jvm ] [Jenkins,
Abner] [gc][ParNew][133] took [3ms]/[1.7s], reclaimed [33.2mb],
leaving [232.9mb] used, max [4.1gb]This is elasticsearch on a single host. No fancy setup, pretty much
straight out of the box, but with 4 GB of memory.Dan
On Jul 1, 12:55 pm, Shay Banon <shay.ba...@elasticsearch.com (http://elasticsearch.com)> wrote:
It failed on one shard, anything in the logs?
On Friday, July 1, 2011 at 7:49 PM, Dan Diephouse wrote:
I'm trying to execute a query to clean out some stuff from our
database, but it's failing. This same query works on another ES
instance that we have but without so much data in it. Any idea why I'm
getting this failure or where I look for more information?curl -XDELETE 'http://host:9200/ion/dantest/_query?
q=fields.domain=dantest.muleion.com (http://dantest.muleion.com)'
{"ok":true,"_indices":{"ion":{"_shards":{"total":5,"successful":
4,"failed":1}}}}Cheers,
Dan
On Sun, 2011-07-03 at 08:39 -0700, Dan Diephouse wrote:
Well, after I wrote my last message I tried this:
curl -XDELETE http://host:9200/ion/dantest
This succeeded. However, while it decreased my number of documents
from 8.5 million to 1.5, my database stayed the exact same size (8
gb)! Any ideas there?I just tried my old query again and it still fails. I'm not sure I
can upload the DB somewhere as it has customer sensitive data in it.
When you delete docs, they are actually just marked as deleted, and are
no longer returned by any queries.
When ES merges segments (the data files that actually store your docs)
then the deleted documents will be removed.
You can force this using the optimize API:
Or you can reindex to a new index, which won't include the old deleted
docs
clint
On Jul 3, 11:42 am, Clinton Gormley clin...@iannounce.co.uk wrote:
When you delete docs, they are actually just marked as deleted, and are
no longer returned by any queries.When ES merges segments (the data files that actually store your docs)
then the deleted documents will be removed.You can force this using the optimize API:Elasticsearch Platform — Find real-time answers at scale | Elastic...
Or you can reindex to a new index, which won't include the old deleted
docs
Thanks. Once I did this:
curl -XPOST 'http://search.muleion.com:9200/ion/_optimize?
only_expunge_deletes=true'
My index went down to the proper size.
Dan
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.