Index missing!

(qwerty1411) #1


i am attempting to get ES working on an EC2 instance with a couchdb back end. I have encounter 3 problems that i am getting no where with.

My first problem is that that i was able to get a river up and going with a node and a few shards going and have "everything" searchable. But then out of the blue(seemingly) when i would curl localhost:9200/data/_status, i would get this error message back:

{"error":"IndexMissingException[[data] missing]","status":404}

i tried searching on that river and it gave the same error message. My first instinct was to delete this river and start a new one, but when i did the command curl -XDELETE localhost:9200/data,i get the same message ({"error":"IndexMissingException[[data] missing]","status":404}), so i can not even delete it. My first issue is this: is there some way to delete this or recover this river?

So, in order to not hold up production of my site, i made a new river. This new river leads me to my other 2 issues:

First, i have roughly 190k docs in my couch, but by doing a /_status on my river, i see only approx 59k. i check out my logs and i see a bunch of docs that have this error:

[2011-11-30 20:46:23,706][DEBUG][action.bulk ] [Hussar] [lr][4] failed to bulk item (index) index {[lr][lr][8b95dbf460cbc93f016df6838e9505c7], source[{"resource_data_...[content data from couchdb here]......
org.elasticsearch.index.mapper.MapperParsingException: Failed to parse [update_timestamp]
at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(
at org.elasticsearch.index.mapper.object.ObjectMapper.serializeValue(
at org.elasticsearch.index.mapper.object.ObjectMapper.parse(
at org.elasticsearch.index.mapper.DocumentMapper.parse(
at org.elasticsearch.index.mapper.DocumentMapper.parse(
at org.elasticsearch.index.shard.service.InternalIndexShard.prepareIndex(
at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(
at java.util.concurrent.ThreadPoolExecutor.runWorker(
at java.util.concurrent.ThreadPoolExecutor$
Caused by: org.elasticsearch.index.mapper.MapperParsingException: failed to parse date field, tried both date format [dateOptionalTime], and timestamp number
at org.elasticsearch.index.mapper.core.DateFieldMapper.parseStringValue(
at org.elasticsearch.index.mapper.core.DateFieldMapper.parseCreateField(
at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(
... 11 more
Caused by: java.lang.IllegalArgumentException: Invalid format: "2010-11-15 09:42:28.747" is malformed at " 09:42:28.747"
at org.elasticsearch.common.joda.time.format.DateTimeFormatter.parseMillis(
at org.elasticsearch.index.mapper.core.DateFieldMapper.parseStringValue(
... 13 more

So my issue with this is that i want to attempt to rescue those docs that did not get indexed somehow, is there a way to attempt to get those docs back into the river?

my last issue is a possible memory leak. On my EC2 instance, i have run out of memory and i can not even edit any files on my instance. i checked the memory allocation and it seems that over 70% of the memory is being used by ES. I suspect that if i can delete my first (and now unused) river that i would be okay, but i can not. Any help or advice on this would be greatly appriciated.

i would rather not do a re-install due to time constraints, but if that is the only option, i guess that is why they make coffee....

(system) #2