TypeMissingException when indexing with 0.15.0-snapshot


(Ryan) #1

while testing a fix for the issue i reported here:
http://groups.google.com/a/elasticsearch.com/group/users/browse_thread/thread/b79c6e1f2ad763c4#

psuedocode:

  • delete existing index
  • create new index with indexing options
  • update index with custom mapping settings
  • index all test data
  • force index refresh
  • perform search

executable testcase available here:

exception:
org.elasticsearch.indices.TypeMissingException: [friends]
type[friends] missing
at
org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGetAction.java:
87)
at
org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGetAction.java:
59)
at
org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction
$AsyncSingleAction$1.run(TransportShardSingleOperationAction.java:130)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)


(Ryan) #2

I've published a simple gist that exercises the issue here:

On Feb 2, 2:01 pm, Ryan ryan.son...@gmail.com wrote:

while testing a fix for the issue i reported here:http://groups.google.com/a/elasticsearch.com/group/users/browse_threa...

psuedocode:

  • delete existing index
  • create new index with indexing options
  • update index with custom mapping settings
  • index all test data
  • force index refresh
  • perform search

executable testcase available here:https://github.com/wireframe/elastic_searchable/blob/master/test/test...

exception:
org.elasticsearch.indices.TypeMissingException: [friends]
type[friends] missing
at
org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGet Action.java:
87)
at
org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGet Action.java:
59)
at
org.elasticsearch.action.support.single.shard.TransportShardSingleOperation Action
$AsyncSingleAction$1.run(TransportShardSingleOperationAction.java:130)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)


(Shay Banon) #3

Can you recreate it with a curl request?
On Wednesday, February 2, 2011 at 10:12 PM, Ryan wrote:

I've published a simple gist that exercises the issue here:
https://gist.github.com/808316

On Feb 2, 2:01 pm, Ryan ryan.son...@gmail.com wrote:

while testing a fix for the issue i reported here:http://groups.google.com/a/elasticsearch.com/group/users/browse_threa...

psuedocode:

  • delete existing index
  • create new index with indexing options
  • update index with custom mapping settings
  • index all test data
  • force index refresh
  • perform search

executable testcase available here:https://github.com/wireframe/elastic_searchable/blob/master/test/test...

exception:
org.elasticsearch.indices.TypeMissingException: [friends]
type[friends] missing
at
org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGet Action.java:
87)
at
org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGet Action.java:
59)
at
org.elasticsearch.action.support.single.shard.TransportShardSingleOperation Action
$AsyncSingleAction$1.run(TransportShardSingleOperationAction.java:130)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)


(Ryan) #4

excellent idea. my curl script is unable to reproduce the issue which
means that this is most likely an issue with rubberband sigh

On Feb 2, 2:43 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

Can you recreate it with a curl request?

On Wednesday, February 2, 2011 at 10:12 PM, Ryan wrote:

I've published a simple gist that exercises the issue here:
https://gist.github.com/808316

On Feb 2, 2:01 pm, Ryan ryan.son...@gmail.com wrote:

while testing a fix for the issue i reported here:http://groups.google.com/a/elasticsearch.com/group/users/browse_threa...

psuedocode:

  • delete existing index
  • create new index with indexing options
  • update index with custom mapping settings
  • index all test data
  • force index refresh
  • perform search

executable testcase available here:https://github.com/wireframe/elastic_searchable/blob/master/test/test...

exception:
org.elasticsearch.indices.TypeMissingException: [friends]
type[friends] missing
at
org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGet Action.java:
87)
at
org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGet Action.java:
59)
at
org.elasticsearch.action.support.single.shard.TransportShardSingleOperation Action
$AsyncSingleAction$1.run(TransportShardSingleOperationAction.java:130)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)


(Shay Banon) #5

Don't want to complicate things :), but it might still be an issue with ES as probably rubberband sends the requests faster then the curl script. If you add delays between operations, does it still happen?
On Thursday, February 3, 2011 at 12:01 AM, Ryan wrote:

excellent idea. my curl script is unable to reproduce the issue which
means that this is most likely an issue with rubberband sigh
https://gist.github.com/808546

On Feb 2, 2:43 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

Can you recreate it with a curl request?

On Wednesday, February 2, 2011 at 10:12 PM, Ryan wrote:

I've published a simple gist that exercises the issue here:
https://gist.github.com/808316

On Feb 2, 2:01 pm, Ryan ryan.son...@gmail.com wrote:

while testing a fix for the issue i reported here:http://groups.google.com/a/elasticsearch.com/group/users/browse_threa...

psuedocode:

  • delete existing index
  • create new index with indexing options
  • update index with custom mapping settings
  • index all test data
  • force index refresh
  • perform search

executable testcase available here:https://github.com/wireframe/elastic_searchable/blob/master/test/test...

exception:
org.elasticsearch.indices.TypeMissingException: [friends]
type[friends] missing
at
org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGet Action.java:
87)
at
org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGet Action.java:
59)
at
org.elasticsearch.action.support.single.shard.TransportShardSingleOperation Action
$AsyncSingleAction$1.run(TransportShardSingleOperationAction.java:130)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)


(Ryan) #6

After spending the last day ripping out rubberband and using straight
up ruby requests, I'm still able to reproduce the issue, but i've also
learned more about how to more-correctly integrate with elasticsearch.

My workaround for now is to not delete the entire index between test
runs. removing the mapping + documents and reindexing my test
fixtures is really all i need to do. Coming from a solr background,
the elasticsearch concept of one index with multiple types is easy to
overlook. i'm so used to nuking and repopulating the index between
test runs. any other recommendations or gotcha's for using
elasticsearch in a test environment would be appreciated!

Thanks again for your time.

On Feb 2, 4:51 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

Don't want to complicate things :), but it might still be an issue with ES as probably rubberband sends the requests faster then the curl script. If you add delays between operations, does it still happen?

On Thursday, February 3, 2011 at 12:01 AM, Ryan wrote:

excellent idea. my curl script is unable to reproduce the issue which
means that this is most likely an issue with rubberband sigh
https://gist.github.com/808546

On Feb 2, 2:43 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

Can you recreate it with a curl request?

On Wednesday, February 2, 2011 at 10:12 PM, Ryan wrote:

I've published a simple gist that exercises the issue here:
https://gist.github.com/808316

On Feb 2, 2:01 pm, Ryan ryan.son...@gmail.com wrote:

while testing a fix for the issue i reported here:http://groups.google.com/a/elasticsearch.com/group/users/browse_threa...

psuedocode:

  • delete existing index
  • create new index with indexing options
  • update index with custom mapping settings
  • index all test data
  • force index refresh
  • perform search

executable testcase available here:https://github.com/wireframe/elastic_searchable/blob/master/test/test...

exception:
org.elasticsearch.indices.TypeMissingException: [friends]
type[friends] missing
at
org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGet Action.java:
87)
at
org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGet Action.java:
59)
at
org.elasticsearch.action.support.single.shard.TransportShardSingleOperation Action
$AsyncSingleAction$1.run(TransportShardSingleOperationAction.java:130)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)


(Shay Banon) #7

Heya,

In elasticsearch tests I delete an index and create it each time. I think that we should solve this problem, lemme ping you back when I have a solution. I think, for a testing perspective, deleting an index and creating it is the correct approach, so its a shame you won't be able to do it.

-shay.banon
On Thursday, February 3, 2011 at 8:40 PM, Ryan wrote:

After spending the last day ripping out rubberband and using straight
up ruby requests, I'm still able to reproduce the issue, but i've also
learned more about how to more-correctly integrate with elasticsearch.

My workaround for now is to not delete the entire index between test
runs. removing the mapping + documents and reindexing my test
fixtures is really all i need to do. Coming from a solr background,
the elasticsearch concept of one index with multiple types is easy to
overlook. i'm so used to nuking and repopulating the index between
test runs. any other recommendations or gotcha's for using
elasticsearch in a test environment would be appreciated!

Thanks again for your time.

On Feb 2, 4:51 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

Don't want to complicate things :), but it might still be an issue with ES as probably rubberband sends the requests faster then the curl script. If you add delays between operations, does it still happen?

On Thursday, February 3, 2011 at 12:01 AM, Ryan wrote:

excellent idea. my curl script is unable to reproduce the issue which
means that this is most likely an issue with rubberband sigh
https://gist.github.com/808546

On Feb 2, 2:43 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

Can you recreate it with a curl request?

On Wednesday, February 2, 2011 at 10:12 PM, Ryan wrote:

I've published a simple gist that exercises the issue here:
https://gist.github.com/808316

On Feb 2, 2:01 pm, Ryan ryan.son...@gmail.com wrote:

while testing a fix for the issue i reported here:http://groups.google.com/a/elasticsearch.com/group/users/browse_threa...

psuedocode:

  • delete existing index
  • create new index with indexing options
  • update index with custom mapping settings
  • index all test data
  • force index refresh
  • perform search

executable testcase available here:https://github.com/wireframe/elastic_searchable/blob/master/test/test...

exception:
org.elasticsearch.indices.TypeMissingException: [friends]
type[friends] missing
at
org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGet Action.java:
87)
at
org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGet Action.java:
59)
at
org.elasticsearch.action.support.single.shard.TransportShardSingleOperation Action
$AsyncSingleAction$1.run(TransportShardSingleOperationAction.java:130)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)


(Shay Banon) #8

I just pushed an improvement to the introduction of a type in elasticsearch through an index operation. Can you give it a go and see if it solves your problem? (I can't recreate your problem locally...).
On Thursday, February 3, 2011 at 8:47 PM, Shay Banon wrote:
Heya,

In elasticsearch tests I delete an index and create it each time. I think that we should solve this problem, lemme ping you back when I have a solution. I think, for a testing perspective, deleting an index and creating it is the correct approach, so its a shame you won't be able to do it.

-shay.banon
On Thursday, February 3, 2011 at 8:40 PM, Ryan wrote:

After spending the last day ripping out rubberband and using straight
up ruby requests, I'm still able to reproduce the issue, but i've also
learned more about how to more-correctly integrate with elasticsearch.

My workaround for now is to not delete the entire index between test
runs. removing the mapping + documents and reindexing my test
fixtures is really all i need to do. Coming from a solr background,
the elasticsearch concept of one index with multiple types is easy to
overlook. i'm so used to nuking and repopulating the index between
test runs. any other recommendations or gotcha's for using
elasticsearch in a test environment would be appreciated!

Thanks again for your time.

On Feb 2, 4:51 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

Don't want to complicate things :), but it might still be an issue with ES as probably rubberband sends the requests faster then the curl script. If you add delays between operations, does it still happen?

On Thursday, February 3, 2011 at 12:01 AM, Ryan wrote:

excellent idea. my curl script is unable to reproduce the issue which
means that this is most likely an issue with rubberband sigh
https://gist.github.com/808546

On Feb 2, 2:43 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

Can you recreate it with a curl request?

On Wednesday, February 2, 2011 at 10:12 PM, Ryan wrote:

I've published a simple gist that exercises the issue here:
https://gist.github.com/808316

On Feb 2, 2:01 pm, Ryan ryan.son...@gmail.com wrote:

while testing a fix for the issue i reported here:http://groups.google.com/a/elasticsearch.com/group/users/browse_threa...

psuedocode:

  • delete existing index
  • create new index with indexing options
  • update index with custom mapping settings
  • index all test data
  • force index refresh
  • perform search

executable testcase available here:https://github.com/wireframe/elastic_searchable/blob/master/test/test...

exception:
org.elasticsearch.indices.TypeMissingException: [friends]
type[friends] missing
at
org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGet Action.java:
87)
at
org.elasticsearch.action.get.TransportGetAction.shardOperation(TransportGet Action.java:
59)
at
org.elasticsearch.action.support.single.shard.TransportShardSingleOperation Action
$AsyncSingleAction$1.run(TransportShardSingleOperationAction.java:130)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)


(system) #9