How does rubberband works when you specify a type? I think the problem might be the way new types are introduced in ES when introduced through the first index request.
Basically, the index request gets hashed to one of the shards and executed there, if this index request resulted in the introduction of a new type, then in the background, an introduce mapping operation it sent to the ES cluster master node, which then broadcast it to all the rest of the nodes. This might explain why you see it (and probably I don't, since I have a faster/slower machine 
).
I'm not sure how to solve it, I can basically delay the index response from coming back if it introduced a new type until the master tells it that it has been introduced, but I am not sure its a good solution, need to think about it.
To verify that my assumption is correct, you can add a sleep right after the index operation of a second, and it should not occur anymore.
-shay.banon
On Wednesday, February 2, 2011 at 11:31 PM, Ryan wrote:
it fails ~30% of the time on my instance (single node setup with a
fresh install of 0.15.snapshot). rerunning the script several times
is really easy for me to reproduce the issue.
the exception generated is:
[2011-02-02 14:16:16,316][DEBUG][action.search.type ] [Firestar]
[foo][0], node[3ZBuCTGgRC-U1jLCBEOxXw], [P], s[STARTED]: Failed to
execute [org.elasticsearch.action.search.SearchRequest@7f77e2f]
org.elasticsearch.search.query.QueryPhaseExecutionException: [foo][0]:
query[_all:foo],from[0],size[10]: Query Failed [Failed to execute main
query]
at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:
194)
at
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:
173)
at
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:
131)
at
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction
$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:
76)
at org.elasticsearch.action.search.type.TransportSearchTypeAction
$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:194)
at org.elasticsearch.action.search.type.TransportSearchTypeAction
$BaseAsyncAction.access$000(TransportSearchTypeAction.java:78)
at org.elasticsearch.action.search.type.TransportSearchTypeAction
$BaseAsyncAction$1.run(TransportSearchTypeAction.java:153)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: org.elasticsearch.indices.TypeMissingException: [foo]
type[bar] missing
at
org.elasticsearch.index.mapper.MapperService.typesFilter(MapperService.java:
243)
at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:
163)
... 9 more
On Feb 2, 2:51 pm, Shay Banon shay.ba...@elasticsearch.com wrote:
I've just ran it, and it ran fine (on latest master). I ran it several times on a two node cluster running locally on my machine. Which exception do you get with this test script?
On Wednesday, February 2, 2011 at 10:12 PM, Ryan wrote:
as a followup, here's a simple gist that reproduces the issue with
0.14.4 and 0.15.snapshot
teardown/recreate index for testcases · GitHub
On Feb 2, 1:10 pm, Shay Banon shay.ba...@elasticsearch.com wrote:
By the way, what does the project do?
Also, any chance someone can work on rubberband to make it use Keep Alive with HTTP?
On Wednesday, February 2, 2011 at 9:08 PM, Shay Banon wrote:
This should hopefully be better on 0.15, your search request hit a node that was not updated with the mappings yet. In maser (0.15), it waits for the other nodes to ack the fact that a mapping was updated on them.
On Wednesday, February 2, 2011 at 8:53 PM, Ryan wrote:
I'm trying to write unit tests that setup/teardown indexes with test
data. I'm using the REST API (ruby, rubberband, elastic_searchable)
and occasionally it errors out with a DocumentMapperNotFoundException
exception.
psuedocode:
- delete existing index
- create new index with indexing options
- update index with custom mapping settings
- index all test data
- force index refresh
- perform search
stacktrace:
Elasticsearch::RequestError: (500)
{"error":"DocumentMapperNotFoundException[No mapper found for type
[friends]]"}
/opt/ruby-enterprise-1.8.7-2010.02/lib/ruby/gems/1.8/gems/
rubberband-0.0.6/lib/elasticsearch/transport/base_protocol.rb:156:in
handle_error' /opt/ruby-enterprise-1.8.7-2010.02/lib/ruby/gems/1.8/gems/ rubberband-0.0.6/lib/elasticsearch/transport/http.rb:43:in request'
/opt/ruby-enterprise-1.8.7-2010.02/lib/ruby/gems/1.8/gems/
rubberband-0.0.6/lib/elasticsearch/transport/base_protocol.rb:16:in
get' /opt/ruby-enterprise-1.8.7-2010.02/lib/ruby/gems/1.8/gems/ rubberband-0.0.6/lib/elasticsearch/client/abstract_client.rb:31:in send'
/opt/ruby-enterprise-1.8.7-2010.02/lib/ruby/gems/1.8/gems/
rubberband-0.0.6/lib/elasticsearch/client/abstract_client.rb:31:in
execute' /opt/ruby-enterprise-1.8.7-2010.02/lib/ruby/gems/1.8/gems/ rubberband-0.0.6/lib/elasticsearch/client/retrying_client.rb:69:in execute'
/opt/ruby-enterprise-1.8.7-2010.02/lib/ruby/gems/1.8/gems/
rubberband-0.0.6/lib/elasticsearch/client/index.rb:37:in get' ./test/test_elastic_searchable.rb:182:in __bind_1296672565_140826'
/opt/ruby-enterprise-1.8.7-2010.02/lib/ruby/gems/1.8/gems/
shoulda-2.11.3/lib/shoulda/context.rb:382:in call' /opt/ruby-enterprise-1.8.7-2010.02/lib/ruby/gems/1.8/gems/ shoulda-2.11.3/lib/shoulda/context.rb:382:in test: activerecord class
with :json=>{} creating index should index json with configuration. '
/opt/ruby-enterprise-1.8.7-2010.02/lib/ruby/gems/1.8/gems/
mocha-0.9.8/lib/mocha/integration/test_unit/
ruby_version_186_and_above.rb:19:in __send__' /opt/ruby-enterprise-1.8.7-2010.02/lib/ruby/gems/1.8/gems/ mocha-0.9.8/lib/mocha/integration/test_unit/ ruby_version_186_and_above.rb:19:in run'
executable testcase available here:
https://github.com/wireframe/elastic_searchable/blob/master/test/test...
any ideas on how to resolve?