Exception from one shard - ArrayIndexOutOfBoundsException


(Bryan Green) #1

Hi folks,

Here is the query I run against this url
'system,library,serc/statustype/_search'
{
"from": 0,
"size": 999,
"sort": {
"order_no": {
"order": "asc"
}
},
"filter": {
"bool": {
"must": [
{
"or": [
{
"term": {
"value": "claimed"
}
},
{
"term": {
"closed_ind": "n"
}
}
]
},
{
"term": {
"item_input_ind": "y"
}
}
]
}
}
}

Here is the exception trace I receive.

[2011-10-16 11:43:29,729][DEBUG][action.search.type ] [Puff Adder]
[serc][1], node[Elt9nySKTtSglTXz9oSYcw], [P], s[STARTED]: Failed to execute
[org.elasticsearch.action.search.SearchRequest@79f3878b]
org.elasticsearch.search.query.QueryPhaseExecutionException: [serc][1]:
query[ConstantScore(:)],from[0],size[999],sort[<custom:"order_no":
org.elasticsearch.index.field.data.longs.LongFieldDataType$1@6e37715c>]:
Query Failed [Failed to execute main query]
at
org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:221)
at
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:234)
at
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:134)
at
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:204)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:191)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:177)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.ArrayIndexOutOfBoundsException

I am running 0.17.8. Anybody know what this is? Thought I saw some
correspondence about this exception last few weeks.

I don't think I even have a result in this particular index for the query to
pull back. I've also had issues with the "order_no" custom sort but after
removing the sorting I still get this same error. Error does not happen
removing the serc index.

Here's some of the details:
Query length: 2.17605ms
took 1 timed_out false _shards total 6 successful 5 failed 1 hits total
5 max_score false

And my setup...
gateway.type: local
gateway.recover_after_nodes: 1
gateway.recover_after_time: 5m
gateway.expected_nodes: 1

index.gateway.snapshot_interval : 4s
index.gateway.snapshot_on_close: true
index.store.type: fs
index.number_of_shards : 2
index.number_of_replicas : 0

Open to any suggestions. Thank you for your time.
-Bryan Green


(Bryan Green) #2

I went ahead and dumped my indexes and brought all of the data back in and
things seem to be working now.

Any ideas what could have happened here or prevention measures for the
future?


(Shay Banon) #3

This smells like this bug:
https://github.com/elasticsearch/elasticsearch/issues/1400, fixed in 0.17
branch and master. Will probably release 0.17.9 sometime this week.

On Sun, Oct 16, 2011 at 9:29 PM, Bryan Green bryogreen@gmail.com wrote:

I went ahead and dumped my indexes and brought all of the data back in and
things seem to be working now.

Any ideas what could have happened here or prevention measures for the
future?


(Bryan Green) #4

Interesting, I should have tried w/o the bool filter.

I went back to the old indexes (directory backup) and the failure went away
went I removed the 'bool' filter. I think 1400 is the rat.

Thank you.

FYI
There is an error in the json in the guide for the _cache version of "or"
here:
http://www.elasticsearch.org/guide/reference/query-dsl/or-filter.html

Parse error on line 9:
... "or": "filters": [
-----------------------^
Expecting '}', ',', ']'


(Kenneth Loafman-2) #5

When was this fixed in master? We started seeing this too, from the Friday
morning master.

...Ken

On Sun, Oct 16, 2011 at 3:38 PM, Bryan Green bryogreen@gmail.com wrote:

Interesting, I should have tried w/o the bool filter.

I went back to the old indexes (directory backup) and the failure went away
went I removed the 'bool' filter. I think 1400 is the rat.

Thank you.

FYI
There is an error in the json in the guide for the _cache version of "or"
here:
http://www.elasticsearch.org/guide/reference/query-dsl/or-filter.html

Parse error on line 9:
... "or": "filters": [
-----------------------^
Expecting '}', ',', ']'


(Shay Banon) #6

It was fixed on Saturday, so you don't have it if you built master on
friday...

On Sun, Oct 16, 2011 at 11:16 PM, Kenneth Loafman kenneth@loafman.comwrote:

When was this fixed in master? We started seeing this too, from the Friday
morning master.

...Ken

On Sun, Oct 16, 2011 at 3:38 PM, Bryan Green bryogreen@gmail.com wrote:

Interesting, I should have tried w/o the bool filter.

I went back to the old indexes (directory backup) and the failure went
away went I removed the 'bool' filter. I think 1400 is the rat.

Thank you.

FYI
There is an error in the json in the guide for the _cache version of "or"
here:
http://www.elasticsearch.org/guide/reference/query-dsl/or-filter.html

Parse error on line 9:
... "or": "filters": [
-----------------------^
Expecting '}', ',', ']'


(system) #7