Unit testing and local nodes


(pzelnip) #1

I'm trying to setup a unit test for a module I wrote which uses
ElasticSearch to index some documents. Internally to this module a
node & client are created, and the regular Java API calls are used for
creating the index, indexing documents, etc. They all seem to work
correctly when connecting to an already running "global" cluster
running on another machine.

Of course though, I'd like to keep my unit test "contained" to the
machine the test is running on. The docs for ES (http://
www.elasticsearch.org/guide/reference/java-api/client.html) seem to
indicate you do this by setting the Node object to be a "local" node
by the local() method on the NodeBuilder object used to create the
Node.

What I expect to happen is that the node creation code should go from
(in the normal use):

node = nodeBuilder().clusterName(clusterName).node();

to (in the unit test):

node = nodeBuilder().clusterName(clusterName).local(true).node();

And the rest of the code in my module under test should work unchanged
after being supplied with this node. Apparently I am mistaken in this
belief.

If I do the above (append a call to .local(true) to the builder), then
my code which waits for the cluster to go from RED status to non-red
status times out after 50 seconds, and I fail to index.

If I drop the clusterName() & retain the .local() then indexing
completes, but no data seems to be stored (doing a query on the
corresponding client returns no hits).

Am I missing something about the use and purpose of a "local" node?
If this isn't how it's intended to be used, then does anyone have any
tips/suggestions on how to test code which uses ES for indexing? I
suppose I could connect to the global cluster, create a "dummy" test
index, then delete it upon test termination, but this seems dangerous
(what if test fails to finish, or goes haywire?) and creates a
seemingly unnecessary coupling between my global ES cluster and the
machine running a test.

In case it matters, this is running ElasticSearch 0.14.4.

Thanks.


(Shay Banon) #2

Running the local node is aimed exactly at unit tests, and actually, elasticsearch has a lot of tests internally that use it.

Maybe the reason why your code hangs when waiting for a cluster health is that you wait for a GREEN health state, where with a single node and replica count > 0 you will never get.

Dropping the cluster name should not change anything. How do you get the client? Do you use node.client()? (which you should).
On Tuesday, March 22, 2011 at 11:47 PM, Adam Parkin wrote:

I'm trying to setup a unit test for a module I wrote which uses
ElasticSearch to index some documents. Internally to this module a
node & client are created, and the regular Java API calls are used for
creating the index, indexing documents, etc. They all seem to work
correctly when connecting to an already running "global" cluster
running on another machine.

Of course though, I'd like to keep my unit test "contained" to the
machine the test is running on. The docs for ES (http://
www.elasticsearch.org/guide/reference/java-api/client.html) seem to
indicate you do this by setting the Node object to be a "local" node
by the local() method on the NodeBuilder object used to create the
Node.

What I expect to happen is that the node creation code should go from
(in the normal use):

node = nodeBuilder().clusterName(clusterName).node();

to (in the unit test):

node = nodeBuilder().clusterName(clusterName).local(true).node();

And the rest of the code in my module under test should work unchanged
after being supplied with this node. Apparently I am mistaken in this
belief.

If I do the above (append a call to .local(true) to the builder), then
my code which waits for the cluster to go from RED status to non-red
status times out after 50 seconds, and I fail to index.

If I drop the clusterName() & retain the .local() then indexing
completes, but no data seems to be stored (doing a query on the
corresponding client returns no hits).

Am I missing something about the use and purpose of a "local" node?
If this isn't how it's intended to be used, then does anyone have any
tips/suggestions on how to test code which uses ES for indexing? I
suppose I could connect to the global cluster, create a "dummy" test
index, then delete it upon test termination, but this seems dangerous
(what if test fails to finish, or goes haywire?) and creates a
seemingly unnecessary coupling between my global ES cluster and the
machine running a test.

In case it matters, this is running ElasticSearch 0.14.4.

Thanks.


(Curtis Caravone) #3

Here's what I do:

  1. Delete any old existing data directory that a previous run created
  2. Create node with .local(true)
  3. Use client.admin().cluster().health() to wait for yellow status
    (waiting for green status seems to hang)
  4. After indexing, use client.admin().indices().refresh() to ensure changes
    are visible
  5. Do queries to check for expected data (if you skip step 2 (waiting for
    yellow status) you might see no data here, even after the refresh in step 3)

Curtis

On Tue, Mar 22, 2011 at 2:47 PM, Adam Parkin pzelnip@gmail.com wrote:

I'm trying to setup a unit test for a module I wrote which uses
ElasticSearch to index some documents. Internally to this module a
node & client are created, and the regular Java API calls are used for
creating the index, indexing documents, etc. They all seem to work
correctly when connecting to an already running "global" cluster
running on another machine.

Of course though, I'd like to keep my unit test "contained" to the
machine the test is running on. The docs for ES (http://
www.elasticsearch.org/guide/reference/java-api/client.html) seem to
indicate you do this by setting the Node object to be a "local" node
by the local() method on the NodeBuilder object used to create the
Node.

What I expect to happen is that the node creation code should go from
(in the normal use):

node = nodeBuilder().clusterName(clusterName).node();

to (in the unit test):

node = nodeBuilder().clusterName(clusterName).local(true).node();

And the rest of the code in my module under test should work unchanged
after being supplied with this node. Apparently I am mistaken in this
belief.

If I do the above (append a call to .local(true) to the builder), then
my code which waits for the cluster to go from RED status to non-red
status times out after 50 seconds, and I fail to index.

If I drop the clusterName() & retain the .local() then indexing
completes, but no data seems to be stored (doing a query on the
corresponding client returns no hits).

Am I missing something about the use and purpose of a "local" node?
If this isn't how it's intended to be used, then does anyone have any
tips/suggestions on how to test code which uses ES for indexing? I
suppose I could connect to the global cluster, create a "dummy" test
index, then delete it upon test termination, but this seems dangerous
(what if test fails to finish, or goes haywire?) and creates a
seemingly unnecessary coupling between my global ES cluster and the
machine running a test.

In case it matters, this is running ElasticSearch 0.14.4.

Thanks.


(Shay Banon) #4

First, I suggest people use the prepareXXX API method calls, they are much nicer :).
On Wednesday, March 23, 2011 at 12:00 AM, Curtis Caravone wrote:

Here's what I do:

  1. Delete any old existing data directory that a previous run created
  2. Create node with .local(true)
  3. Use client.admin().cluster().health() to wait for yellow status (waiting for green status seems to hang)

It does not hang, it will simply wait for the default timeout (30s), and then return with a flag indicating that it has timed out.

  1. After indexing, use client.admin().indices().refresh() to ensure changes are visible
  2. Do queries to check for expected data (if you skip step 2 (waiting for yellow status) you might see no data here, even after the refresh in step 3)
    You should see changes. Which version are you using? There was a bug relating to that which was fixed in 0.15.

Curtis

On Tue, Mar 22, 2011 at 2:47 PM, Adam Parkin pzelnip@gmail.com wrote:

I'm trying to setup a unit test for a module I wrote which uses
ElasticSearch to index some documents. Internally to this module a
node & client are created, and the regular Java API calls are used for
creating the index, indexing documents, etc. They all seem to work
correctly when connecting to an already running "global" cluster
running on another machine.

Of course though, I'd like to keep my unit test "contained" to the
machine the test is running on. The docs for ES (http://
www.elasticsearch.org/guide/reference/java-api/client.html) seem to
indicate you do this by setting the Node object to be a "local" node
by the local() method on the NodeBuilder object used to create the
Node.

What I expect to happen is that the node creation code should go from
(in the normal use):

node = nodeBuilder().clusterName(clusterName).node();

to (in the unit test):

node = nodeBuilder().clusterName(clusterName).local(true).node();

And the rest of the code in my module under test should work unchanged
after being supplied with this node. Apparently I am mistaken in this
belief.

If I do the above (append a call to .local(true) to the builder), then
my code which waits for the cluster to go from RED status to non-red
status times out after 50 seconds, and I fail to index.

If I drop the clusterName() & retain the .local() then indexing
completes, but no data seems to be stored (doing a query on the
corresponding client returns no hits).

Am I missing something about the use and purpose of a "local" node?
If this isn't how it's intended to be used, then does anyone have any
tips/suggestions on how to test code which uses ES for indexing? I
suppose I could connect to the global cluster, create a "dummy" test
index, then delete it upon test termination, but this seems dangerous
(what if test fails to finish, or goes haywire?) and creates a
seemingly unnecessary coupling between my global ES cluster and the
machine running a test.

In case it matters, this is running ElasticSearch 0.14.4.

Thanks.


(Curtis Caravone) #5

version was somewhere between 0.15.0 and 0.15.2. I'll let you know if I can
reproduce it in 0.15.2.

Curtis

On Tue, Mar 22, 2011 at 3:03 PM, Shay Banon shay.banon@elasticsearch.comwrote:

First, I suggest people use the prepareXXX API method calls, they are
much nicer :).

On Wednesday, March 23, 2011 at 12:00 AM, Curtis Caravone wrote:

Here's what I do:

  1. Delete any old existing data directory that a previous run created
  2. Create node with .local(true)
  3. Use client.admin().cluster().health() to wait for yellow status
    (waiting for green status seems to hang)

It does not hang, it will simply wait for the default timeout (30s), and
then return with a flag indicating that it has timed out.

  1. After indexing, use client.admin().indices().refresh() to ensure
    changes are visible
  2. Do queries to check for expected data (if you skip step 2 (waiting for
    yellow status) you might see no data here, even after the refresh in step 3)

You should see changes. Which version are you using? There was a bug
relating to that which was fixed in 0.15.

Curtis

On Tue, Mar 22, 2011 at 2:47 PM, Adam Parkin pzelnip@gmail.com wrote:

I'm trying to setup a unit test for a module I wrote which uses
ElasticSearch to index some documents. Internally to this module a
node & client are created, and the regular Java API calls are used for
creating the index, indexing documents, etc. They all seem to work
correctly when connecting to an already running "global" cluster
running on another machine.

Of course though, I'd like to keep my unit test "contained" to the
machine the test is running on. The docs for ES (http://
www.elasticsearch.org/guide/reference/java-api/client.html) seem to
indicate you do this by setting the Node object to be a "local" node
by the local() method on the NodeBuilder object used to create the
Node.

What I expect to happen is that the node creation code should go from
(in the normal use):

node = nodeBuilder().clusterName(clusterName).node();

to (in the unit test):

node = nodeBuilder().clusterName(clusterName).local(true).node();

And the rest of the code in my module under test should work unchanged
after being supplied with this node. Apparently I am mistaken in this
belief.

If I do the above (append a call to .local(true) to the builder), then
my code which waits for the cluster to go from RED status to non-red
status times out after 50 seconds, and I fail to index.

If I drop the clusterName() & retain the .local() then indexing
completes, but no data seems to be stored (doing a query on the
corresponding client returns no hits).

Am I missing something about the use and purpose of a "local" node?
If this isn't how it's intended to be used, then does anyone have any
tips/suggestions on how to test code which uses ES for indexing? I
suppose I could connect to the global cluster, create a "dummy" test
index, then delete it upon test termination, but this seems dangerous
(what if test fails to finish, or goes haywire?) and creates a
seemingly unnecessary coupling between my global ES cluster and the
machine running a test.

In case it matters, this is running ElasticSearch 0.14.4.

Thanks.


(pzelnip) #6

I don't wait for Green, but rather yellow.

And yes, the Client is created as:

Client client = node.client();

As for the prepareXXX calls, I'm not familiar with them (they're not
documented on the site so far as I could see), is there a starting
point I should look at to explore them?

I'll try Curtis' suggested ideas along with trying v0.15+ and report
back tomorrow.

Thanks.

On Mar 22, 2:56 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

Running the local node is aimed exactly at unit tests, and actually, elasticsearch has a lot of tests internally that use it.

Maybe the reason why your code hangs when waiting for a cluster health is that you wait for a GREEN health state, where with a single node and replica count > 0 you will never get.

Dropping the cluster name should not change anything. How do you get the client? Do you use node.client()? (which you should).

On Tuesday, March 22, 2011 at 11:47 PM, Adam Parkin wrote:

I'm trying to setup a unit test for a module I wrote which uses
ElasticSearch to index some documents. Internally to this module a
node & client are created, and the regular Java API calls are used for
creating the index, indexing documents, etc. They all seem to work
correctly when connecting to an already running "global" cluster
running on another machine.

Of course though, I'd like to keep my unit test "contained" to the
machine the test is running on. The docs for ES (http://
www.elasticsearch.org/guide/reference/java-api/client.html) seem to
indicate you do this by setting the Node object to be a "local" node
by the local() method on the NodeBuilder object used to create the
Node.

What I expect to happen is that the node creation code should go from
(in the normal use):

node = nodeBuilder().clusterName(clusterName).node();

to (in the unit test):

node = nodeBuilder().clusterName(clusterName).local(true).node();

And the rest of the code in my module under test should work unchanged
after being supplied with this node. Apparently I am mistaken in this
belief.

If I do the above (append a call to .local(true) to the builder), then
my code which waits for the cluster to go from RED status to non-red
status times out after 50 seconds, and I fail to index.

If I drop the clusterName() & retain the .local() then indexing
completes, but no data seems to be stored (doing a query on the
corresponding client returns no hits).

Am I missing something about the use and purpose of a "local" node?
If this isn't how it's intended to be used, then does anyone have any
tips/suggestions on how to test code which uses ES for indexing? I
suppose I could connect to the global cluster, create a "dummy" test
index, then delete it upon test termination, but this seems dangerous
(what if test fails to finish, or goes haywire?) and creates a
seemingly unnecessary coupling between my global ES cluster and the
machine running a test.

In case it matters, this is running ElasticSearch 0.14.4.

Thanks.


(Shay Banon) #7

If you can gist a simple sample of the code that fails, I can give it a go.
On Wednesday, March 23, 2011 at 6:17 AM, Adam Parkin wrote:

I don't wait for Green, but rather yellow.

And yes, the Client is created as:

Client client = node.client();

As for the prepareXXX calls, I'm not familiar with them (they're not
documented on the site so far as I could see), is there a starting
point I should look at to explore them?

I'll try Curtis' suggested ideas along with trying v0.15+ and report
back tomorrow.

Thanks.

On Mar 22, 2:56 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

Running the local node is aimed exactly at unit tests, and actually, elasticsearch has a lot of tests internally that use it.

Maybe the reason why your code hangs when waiting for a cluster health is that you wait for a GREEN health state, where with a single node and replica count > 0 you will never get.

Dropping the cluster name should not change anything. How do you get the client? Do you use node.client()? (which you should).

On Tuesday, March 22, 2011 at 11:47 PM, Adam Parkin wrote:

I'm trying to setup a unit test for a module I wrote which uses
ElasticSearch to index some documents. Internally to this module a
node & client are created, and the regular Java API calls are used for
creating the index, indexing documents, etc. They all seem to work
correctly when connecting to an already running "global" cluster
running on another machine.

Of course though, I'd like to keep my unit test "contained" to the
machine the test is running on. The docs for ES (http://
www.elasticsearch.org/guide/reference/java-api/client.html) seem to
indicate you do this by setting the Node object to be a "local" node
by the local() method on the NodeBuilder object used to create the
Node.

What I expect to happen is that the node creation code should go from
(in the normal use):

node = nodeBuilder().clusterName(clusterName).node();

to (in the unit test):

node = nodeBuilder().clusterName(clusterName).local(true).node();

And the rest of the code in my module under test should work unchanged
after being supplied with this node. Apparently I am mistaken in this
belief.

If I do the above (append a call to .local(true) to the builder), then
my code which waits for the cluster to go from RED status to non-red
status times out after 50 seconds, and I fail to index.

If I drop the clusterName() & retain the .local() then indexing
completes, but no data seems to be stored (doing a query on the
corresponding client returns no hits).

Am I missing something about the use and purpose of a "local" node?
If this isn't how it's intended to be used, then does anyone have any
tips/suggestions on how to test code which uses ES for indexing? I
suppose I could connect to the global cluster, create a "dummy" test
index, then delete it upon test termination, but this seems dangerous
(what if test fails to finish, or goes haywire?) and creates a
seemingly unnecessary coupling between my global ES cluster and the
machine running a test.

In case it matters, this is running ElasticSearch 0.14.4.

Thanks.


(Shay Banon) #8

Ohh, and the prepareXXX are the only one that are now documented on the site.
On Wednesday, March 23, 2011 at 11:16 AM, Shay Banon wrote:

If you can gist a simple sample of the code that fails, I can give it a go.
On Wednesday, March 23, 2011 at 6:17 AM, Adam Parkin wrote:

I don't wait for Green, but rather yellow.

And yes, the Client is created as:

Client client = node.client();

As for the prepareXXX calls, I'm not familiar with them (they're not
documented on the site so far as I could see), is there a starting
point I should look at to explore them?

I'll try Curtis' suggested ideas along with trying v0.15+ and report
back tomorrow.

Thanks.

On Mar 22, 2:56 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

Running the local node is aimed exactly at unit tests, and actually, elasticsearch has a lot of tests internally that use it.

Maybe the reason why your code hangs when waiting for a cluster health is that you wait for a GREEN health state, where with a single node and replica count > 0 you will never get.

Dropping the cluster name should not change anything. How do you get the client? Do you use node.client()? (which you should).

On Tuesday, March 22, 2011 at 11:47 PM, Adam Parkin wrote:

I'm trying to setup a unit test for a module I wrote which uses
ElasticSearch to index some documents. Internally to this module a
node & client are created, and the regular Java API calls are used for
creating the index, indexing documents, etc. They all seem to work
correctly when connecting to an already running "global" cluster
running on another machine.

Of course though, I'd like to keep my unit test "contained" to the
machine the test is running on. The docs for ES (http://
www.elasticsearch.org/guide/reference/java-api/client.html) seem to
indicate you do this by setting the Node object to be a "local" node
by the local() method on the NodeBuilder object used to create the
Node.

What I expect to happen is that the node creation code should go from
(in the normal use):

node = nodeBuilder().clusterName(clusterName).node();

to (in the unit test):

node = nodeBuilder().clusterName(clusterName).local(true).node();

And the rest of the code in my module under test should work unchanged
after being supplied with this node. Apparently I am mistaken in this
belief.

If I do the above (append a call to .local(true) to the builder), then
my code which waits for the cluster to go from RED status to non-red
status times out after 50 seconds, and I fail to index.

If I drop the clusterName() & retain the .local() then indexing
completes, but no data seems to be stored (doing a query on the
corresponding client returns no hits).

Am I missing something about the use and purpose of a "local" node?
If this isn't how it's intended to be used, then does anyone have any
tips/suggestions on how to test code which uses ES for indexing? I
suppose I could connect to the global cluster, create a "dummy" test
index, then delete it upon test termination, but this seems dangerous
(what if test fails to finish, or goes haywire?) and creates a
seemingly unnecessary coupling between my global ES cluster and the
machine running a test.

In case it matters, this is running ElasticSearch 0.14.4.

Thanks.


(pzelnip) #9

If you mean for doing indexing & queries, then yeah I already use the
prepareXXX methods. I took your response to indicate that creation of
the node/client has an equivalent prepareXXX set of methods, which are
not on the site. I think I just misunderstood what you originally
meant. :slight_smile:

Anywho: I tried incorporating Curtis' suggestions (most of which I was
already doing, the exception being doing a refresh on the index after
indexing). No change, had same problem (indexing completes, but no
matches on query).

I also tried switching to v0.15.2, and ran locally, and had no
change. Indexing completes, but the query itself returns no
matches.

My testing code indexes a single record which consisted of two fields:
firstname & lastname, which had values "Douglas" and "Adams", and the
ID for the record was (of course) set to 42.

The query code looks like

    SearchResponse response = client.prepareSearch(INDEX_NAME)
            .setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
            .setQuery(fieldQuery("firstname", "Douglas"))
            .addFields("firstname",

"lastname").setFrom(0).setSize(60)
.setExplain(true).execute().actionGet();

And printing the results of the query looks like:

    for (SearchHit sh : response.getHits()) {
        System.out.println("Showing hit: '"
                + (String) sh.field("firstname").getValue() +

"'");
}

Which prints nothing to the screen (though when I ran the same code
without making the node local, I did get "Douglas" printed on the
screen as expected.

I'll try and write up a simple class which exploits the exact behavior
I'm seeing and post back with it.

Thanks.

On Mar 23, 2:17 am, Shay Banon shay.ba...@elasticsearch.com wrote:

Ohh, and the prepareXXX are the only one that are now documented on the site.

On Wednesday, March 23, 2011 at 11:16 AM, Shay Banon wrote:

If you can gist a simple sample of the code that fails, I can give it a go.
On Wednesday, March 23, 2011 at 6:17 AM, Adam Parkin wrote:

I don't wait for Green, but rather yellow.

And yes, the Client is created as:

Client client = node.client();

As for the prepareXXX calls, I'm not familiar with them (they're not
documented on the site so far as I could see), is there a starting
point I should look at to explore them?

I'll try Curtis' suggested ideas along with trying v0.15+ and report
back tomorrow.

Thanks.

On Mar 22, 2:56 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

Running the local node is aimed exactly at unit tests, and actually, elasticsearch has a lot of tests internally that use it.

Maybe the reason why your code hangs when waiting for a cluster health is that you wait for a GREEN health state, where with a single node and replica count > 0 you will never get.

Dropping the cluster name should not change anything. How do you get the client? Do you use node.client()? (which you should).

On Tuesday, March 22, 2011 at 11:47 PM, Adam Parkin wrote:

I'm trying to setup a unit test for a module I wrote which uses
ElasticSearch to index some documents. Internally to this module a
node & client are created, and the regular Java API calls are used for
creating the index, indexing documents, etc. They all seem to work
correctly when connecting to an already running "global" cluster
running on another machine.

Of course though, I'd like to keep my unit test "contained" to the
machine the test is running on. The docs for ES (http://
www.elasticsearch.org/guide/reference/java-api/client.html) seem to
indicate you do this by setting the Node object to be a "local" node
by the local() method on the NodeBuilder object used to create the
Node.

What I expect to happen is that the node creation code should go from
(in the normal use):

node = nodeBuilder().clusterName(clusterName).node();

to (in the unit test):

node = nodeBuilder().clusterName(clusterName).local(true).node();

And the rest of the code in my module under test should work unchanged
after being supplied with this node. Apparently I am mistaken in this
belief.

If I do the above (append a call to .local(true) to the builder), then
my code which waits for the cluster to go from RED status to non-red
status times out after 50 seconds, and I fail to index.

If I drop the clusterName() & retain the .local() then indexing
completes, but no data seems to be stored (doing a query on the
corresponding client returns no hits).

Am I missing something about the use and purpose of a "local" node?
If this isn't how it's intended to be used, then does anyone have any
tips/suggestions on how to test code which uses ES for indexing? I
suppose I could connect to the global cluster, create a "dummy" test
index, then delete it upon test termination, but this seems dangerous
(what if test fails to finish, or goes haywire?) and creates a
seemingly unnecessary coupling between my global ES cluster and the
machine running a test.

In case it matters, this is running ElasticSearch 0.14.4.

Thanks.


(Shay Banon) #10

A test case would be great. Did you refresh after the indexing is done?
On Wednesday, March 23, 2011 at 6:17 PM, Adam Parkin wrote:

If you mean for doing indexing & queries, then yeah I already use the
prepareXXX methods. I took your response to indicate that creation of
the node/client has an equivalent prepareXXX set of methods, which are
not on the site. I think I just misunderstood what you originally
meant. :slight_smile:

Anywho: I tried incorporating Curtis' suggestions (most of which I was
already doing, the exception being doing a refresh on the index after
indexing). No change, had same problem (indexing completes, but no
matches on query).

I also tried switching to v0.15.2, and ran locally, and had no
change. Indexing completes, but the query itself returns no
matches.

My testing code indexes a single record which consisted of two fields:
firstname & lastname, which had values "Douglas" and "Adams", and the
ID for the record was (of course) set to 42.

The query code looks like

SearchResponse response = client.prepareSearch(INDEX_NAME)
.setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
.setQuery(fieldQuery("firstname", "Douglas"))
.addFields("firstname",
"lastname").setFrom(0).setSize(60)
.setExplain(true).execute().actionGet();

And printing the results of the query looks like:

for (SearchHit sh : response.getHits()) {
System.out.println("Showing hit: '"

  • (String) sh.field("firstname").getValue() +
    "'");
    }

Which prints nothing to the screen (though when I ran the same code
without making the node local, I did get "Douglas" printed on the
screen as expected.

I'll try and write up a simple class which exploits the exact behavior
I'm seeing and post back with it.

Thanks.

On Mar 23, 2:17 am, Shay Banon shay.ba...@elasticsearch.com wrote:

Ohh, and the prepareXXX are the only one that are now documented on the site.

On Wednesday, March 23, 2011 at 11:16 AM, Shay Banon wrote:

If you can gist a simple sample of the code that fails, I can give it a go.
On Wednesday, March 23, 2011 at 6:17 AM, Adam Parkin wrote:

I don't wait for Green, but rather yellow.

And yes, the Client is created as:

Client client = node.client();

As for the prepareXXX calls, I'm not familiar with them (they're not
documented on the site so far as I could see), is there a starting
point I should look at to explore them?

I'll try Curtis' suggested ideas along with trying v0.15+ and report
back tomorrow.

Thanks.

On Mar 22, 2:56 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

Running the local node is aimed exactly at unit tests, and actually, elasticsearch has a lot of tests internally that use it.

Maybe the reason why your code hangs when waiting for a cluster health is that you wait for a GREEN health state, where with a single node and replica count > 0 you will never get.

Dropping the cluster name should not change anything. How do you get the client? Do you use node.client()? (which you should).

On Tuesday, March 22, 2011 at 11:47 PM, Adam Parkin wrote:

I'm trying to setup a unit test for a module I wrote which uses
ElasticSearch to index some documents. Internally to this module a
node & client are created, and the regular Java API calls are used for
creating the index, indexing documents, etc. They all seem to work
correctly when connecting to an already running "global" cluster
running on another machine.

Of course though, I'd like to keep my unit test "contained" to the
machine the test is running on. The docs for ES (http://
www.elasticsearch.org/guide/reference/java-api/client.html) seem to
indicate you do this by setting the Node object to be a "local" node
by the local() method on the NodeBuilder object used to create the
Node.

What I expect to happen is that the node creation code should go from
(in the normal use):

node = nodeBuilder().clusterName(clusterName).node();

to (in the unit test):

node = nodeBuilder().clusterName(clusterName).local(true).node();

And the rest of the code in my module under test should work unchanged
after being supplied with this node. Apparently I am mistaken in this
belief.

If I do the above (append a call to .local(true) to the builder), then
my code which waits for the cluster to go from RED status to non-red
status times out after 50 seconds, and I fail to index.

If I drop the clusterName() & retain the .local() then indexing
completes, but no data seems to be stored (doing a query on the
corresponding client returns no hits).

Am I missing something about the use and purpose of a "local" node?
If this isn't how it's intended to be used, then does anyone have any
tips/suggestions on how to test code which uses ES for indexing? I
suppose I could connect to the global cluster, create a "dummy" test
index, then delete it upon test termination, but this seems dangerous
(what if test fails to finish, or goes haywire?) and creates a
seemingly unnecessary coupling between my global ES cluster and the
machine running a test.

In case it matters, this is running ElasticSearch 0.14.4.

Thanks.


(pzelnip) #11

Here goes:

public void runESTest() {
    final String clusterName = "testcluster";
    final String indexName = "testindex";
    final String typeName = "testtype";

    // delete old data/ directory
    deleteDataDirectory();

    Node node =

nodeBuilder().clusterName(clusterName).local(true).node();
Client client = node.client();

    // create the index
    try {
        waitForReady(client);

client.admin().indices().create(createIndexRequest(indexName))
.actionGet();
} catch (RemoteTransportException e) {
logger.error("Error creating index {}", indexName, e);
System.exit(1);
}

    // index a record
    String id = "42";
    HashMap<String, Object> data = new HashMap<String, Object>();
    data.put("firstname", "Douglas");
    data.put("lastname", "Adams");

    try {
        waitForReady(client);
        client.prepareIndex(indexName, typeName, id)
                .setSource(buildJSON(data)).execute().actionGet();
        waitForRefresh(client, indexName);
    } catch (ElasticSearchException e) {
        logger.error("Failure to index document with ID: {}", id,

e);
} catch (IOException e) {
logger.error("Failure to create JSON to index", e);
}

    // now query for the record.
    waitForReady(client);
    SearchResponse response = client.prepareSearch(indexName)
            .setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
            .setQuery(fieldQuery("firstname", "Douglas"))
            .addFields("firstname",

"lastname").setFrom(0).setSize(60)
.setExplain(true).execute().actionGet();

    System.out.println("Showing hits...");
    for (SearchHit sh : response.getHits()) {
        System.out.println("Showing hit: '"
                + (String) sh.field("firstname").getValue() +

"'");
}
System.out.println("Done Showing hits...");

    // close down client & node
    waitForReady(client);
    client.close();
    node.close();
}

private XContentBuilder buildJSON(HashMap<String, Object> items)
        throws IOException {

    XContentBuilder xcb = jsonBuilder();
    xcb.startObject();

    for (String key : items.keySet())
        xcb.field(key, items.get(key));

    xcb.endObject();
    logger.info("JSON for record: {}", xcb.string());
    return xcb;
}

void waitForReady(Client client) {
    // wait 30 seconds for yellow
    ClusterHealthRequest req = new ClusterHealthRequest().timeout(
            new TimeValue(30,

TimeUnit.SECONDS)).waitForYellowStatus();

    ClusterHealthResponse response =

client.admin().cluster().health(req)
.actionGet();

    if (response.getStatus() == ClusterHealthStatus.RED) {
        logger.error("Cluster failed to reach ready

status...aborting");
throw new RuntimeException(
"ElasticSearch cluster status failed become
ready");
}
}

private static void waitForRefresh(Client client, String

indexName) {
client.admin().indices().refresh(refreshRequest(indexName));
}

This code prints out the Douglas Adams record when I change the
local() call in the NodeBuilder to be false (at least with v0.14.4),
but prints out no records when it is set to true (in both v0.14.4 and
v0.15.2).

Not sure if I'm refreshing the index correctly, so any feedback on the
waitForRefresh() method would be appreciated.

Thanks.

On Mar 23, 9:37 am, Shay Banon shay.ba...@elasticsearch.com wrote:

A test case would be great. Did you refresh after the indexing is done?

On Wednesday, March 23, 2011 at 6:17 PM, Adam Parkin wrote:

If you mean for doing indexing & queries, then yeah I already use the
prepareXXX methods. I took your response to indicate that creation of
the node/client has an equivalent prepareXXX set of methods, which are
not on the site. I think I just misunderstood what you originally
meant. :slight_smile:

Anywho: I tried incorporating Curtis' suggestions (most of which I was
already doing, the exception being doing a refresh on the index after
indexing). No change, had same problem (indexing completes, but no
matches on query).

I also tried switching to v0.15.2, and ran locally, and had no
change. Indexing completes, but the query itself returns no
matches.

My testing code indexes a single record which consisted of two fields:
firstname & lastname, which had values "Douglas" and "Adams", and the
ID for the record was (of course) set to 42.

The query code looks like

SearchResponse response = client.prepareSearch(INDEX_NAME)
.setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
.setQuery(fieldQuery("firstname", "Douglas"))
.addFields("firstname",
"lastname").setFrom(0).setSize(60)
.setExplain(true).execute().actionGet();

And printing the results of the query looks like:

for (SearchHit sh : response.getHits()) {
System.out.println("Showing hit: '"

  • (String) sh.field("firstname").getValue() +
    "'");
    }

Which prints nothing to the screen (though when I ran the same code
without making the node local, I did get "Douglas" printed on the
screen as expected.

I'll try and write up a simple class which exploits the exact behavior
I'm seeing and post back with it.

Thanks.

On Mar 23, 2:17 am, Shay Banon shay.ba...@elasticsearch.com wrote:

Ohh, and the prepareXXX are the only one that are now documented on the site.

On Wednesday, March 23, 2011 at 11:16 AM, Shay Banon wrote:

If you can gist a simple sample of the code that fails, I can give it a go.
On Wednesday, March 23, 2011 at 6:17 AM, Adam Parkin wrote:

I don't wait for Green, but rather yellow.

And yes, the Client is created as:

Client client = node.client();

As for the prepareXXX calls, I'm not familiar with them (they're not
documented on the site so far as I could see), is there a starting
point I should look at to explore them?

I'll try Curtis' suggested ideas along with trying v0.15+ and report
back tomorrow.

Thanks.

On Mar 22, 2:56 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

Running the local node is aimed exactly at unit tests, and actually, elasticsearch has a lot of tests internally that use it.

Maybe the reason why your code hangs when waiting for a cluster health is that you wait for a GREEN health state, where with a single node and replica count > 0 you will never get.

Dropping the cluster name should not change anything. How do you get the client? Do you use node.client()? (which you should).

On Tuesday, March 22, 2011 at 11:47 PM, Adam Parkin wrote:

I'm trying to setup a unit test for a module I wrote which uses
ElasticSearch to index some documents. Internally to this module a
node & client are created, and the regular Java API calls are used for
creating the index, indexing documents, etc. They all seem to work
correctly when connecting to an already running "global" cluster
running on another machine.

Of course though, I'd like to keep my unit test "contained" to the
machine the test is running on. The docs for ES (http://
www.elasticsearch.org/guide/reference/java-api/client.html) seem to
indicate you do this by setting the Node object to be a "local" node
by the local() method on the NodeBuilder object used to create the
Node.

What I expect to happen is that the node creation code should go from
(in the normal use):

node = nodeBuilder().clusterName(clusterName).node();

to (in the unit test):

node = nodeBuilder().clusterName(clusterName).local(true).node();

And the rest of the code in my module under test should work unchanged
after being supplied with this node. Apparently I am mistaken in this
belief.

If I do the above (append a call to .local(true) to the builder), then
my code which waits for the cluster to go from RED status to non-red
status times out after 50 seconds, and I fail to index.

If I drop the clusterName() & retain the .local() then indexing
completes, but no data seems to be stored (doing a query on the
corresponding client returns no hits).

Am I missing something about the use and purpose of a "local" node?
If this isn't how it's intended to be used, then does anyone have any
tips/suggestions on how to test code which uses ES for indexing? I
suppose I could connect to the global cluster, create a "dummy" test
index, then delete it upon test termination, but this seems dangerous
(what if test fails to finish, or goes haywire?) and creates a
seemingly unnecessary coupling between my global ES cluster and the
machine running a test.

In case it matters, this is running ElasticSearch 0.14.4.

Thanks.


(Shay Banon) #12

When refreshing and you wan to make sure the refresh has executed, you should execute and actionGet it: client.admin().indices().prepareRefresh().execute().actionGet().

Also, when you post a sample, gist it.
On Wednesday, March 23, 2011 at 7:11 PM, Adam Parkin wrote:
Here goes:

public void runESTest() {
final String clusterName = "testcluster";
final String indexName = "testindex";
final String typeName = "testtype";

// delete old data/ directory
deleteDataDirectory();

Node node =
nodeBuilder().clusterName(clusterName).local(true).node();
Client client = node.client();

// create the index
try {
waitForReady(client);

client.admin().indices().create(createIndexRequest(indexName))
.actionGet();
} catch (RemoteTransportException e) {
logger.error("Error creating index {}", indexName, e);
System.exit(1);
}

// index a record
String id = "42";
HashMap<String, Object> data = new HashMap<String, Object>();
data.put("firstname", "Douglas");
data.put("lastname", "Adams");

try {
waitForReady(client);
client.prepareIndex(indexName, typeName, id)
.setSource(buildJSON(data)).execute().actionGet();
waitForRefresh(client, indexName);
} catch (ElasticSearchException e) {
logger.error("Failure to index document with ID: {}", id,
e);
} catch (IOException e) {
logger.error("Failure to create JSON to index", e);
}

// now query for the record.
waitForReady(client);
SearchResponse response = client.prepareSearch(indexName)
.setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
.setQuery(fieldQuery("firstname", "Douglas"))
.addFields("firstname",
"lastname").setFrom(0).setSize(60)
.setExplain(true).execute().actionGet();

System.out.println("Showing hits...");
for (SearchHit sh : response.getHits()) {
System.out.println("Showing hit: '"

  • (String) sh.field("firstname").getValue() +
    "'");
    }
    System.out.println("Done Showing hits...");

// close down client & node
waitForReady(client);
client.close();
node.close();
}

private XContentBuilder buildJSON(HashMap<String, Object> items)
throws IOException {

XContentBuilder xcb = jsonBuilder();
xcb.startObject();

for (String key : items.keySet())
xcb.field(key, items.get(key));

xcb.endObject();
logger.info("JSON for record: {}", xcb.string());
return xcb;
}

void waitForReady(Client client) {
// wait 30 seconds for yellow
ClusterHealthRequest req = new ClusterHealthRequest().timeout(
new TimeValue(30,
TimeUnit.SECONDS)).waitForYellowStatus();

ClusterHealthResponse response =
client.admin().cluster().health(req)
.actionGet();

if (response.getStatus() == ClusterHealthStatus.RED) {
logger.error("Cluster failed to reach ready
status...aborting");
throw new RuntimeException(
"ElasticSearch cluster status failed become
ready");
}
}

private static void waitForRefresh(Client client, String
indexName) {
client.admin().indices().refresh(refreshRequest(indexName));
}

This code prints out the Douglas Adams record when I change the
local() call in the NodeBuilder to be false (at least with v0.14.4),
but prints out no records when it is set to true (in both v0.14.4 and
v0.15.2).

Not sure if I'm refreshing the index correctly, so any feedback on the
waitForRefresh() method would be appreciated.

Thanks.

On Mar 23, 9:37 am, Shay Banon shay.ba...@elasticsearch.com wrote:

A test case would be great. Did you refresh after the indexing is done?

On Wednesday, March 23, 2011 at 6:17 PM, Adam Parkin wrote:

If you mean for doing indexing & queries, then yeah I already use the
prepareXXX methods. I took your response to indicate that creation of
the node/client has an equivalent prepareXXX set of methods, which are
not on the site. I think I just misunderstood what you originally
meant. :slight_smile:

Anywho: I tried incorporating Curtis' suggestions (most of which I was
already doing, the exception being doing a refresh on the index after
indexing). No change, had same problem (indexing completes, but no
matches on query).

I also tried switching to v0.15.2, and ran locally, and had no
change. Indexing completes, but the query itself returns no
matches.

My testing code indexes a single record which consisted of two fields:
firstname & lastname, which had values "Douglas" and "Adams", and the
ID for the record was (of course) set to 42.

The query code looks like

SearchResponse response = client.prepareSearch(INDEX_NAME)
.setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
.setQuery(fieldQuery("firstname", "Douglas"))
.addFields("firstname",
"lastname").setFrom(0).setSize(60)
.setExplain(true).execute().actionGet();

And printing the results of the query looks like:

for (SearchHit sh : response.getHits()) {
System.out.println("Showing hit: '"

  • (String) sh.field("firstname").getValue() +
    "'");
    }

Which prints nothing to the screen (though when I ran the same code
without making the node local, I did get "Douglas" printed on the
screen as expected.

I'll try and write up a simple class which exploits the exact behavior
I'm seeing and post back with it.

Thanks.

On Mar 23, 2:17 am, Shay Banon shay.ba...@elasticsearch.com wrote:

Ohh, and the prepareXXX are the only one that are now documented on the site.

On Wednesday, March 23, 2011 at 11:16 AM, Shay Banon wrote:

If you can gist a simple sample of the code that fails, I can give it a go.
On Wednesday, March 23, 2011 at 6:17 AM, Adam Parkin wrote:

I don't wait for Green, but rather yellow.

And yes, the Client is created as:

Client client = node.client();

As for the prepareXXX calls, I'm not familiar with them (they're not
documented on the site so far as I could see), is there a starting
point I should look at to explore them?

I'll try Curtis' suggested ideas along with trying v0.15+ and report
back tomorrow.

Thanks.

On Mar 22, 2:56 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

Running the local node is aimed exactly at unit tests, and actually, elasticsearch has a lot of tests internally that use it.

Maybe the reason why your code hangs when waiting for a cluster health is that you wait for a GREEN health state, where with a single node and replica count > 0 you will never get.

Dropping the cluster name should not change anything. How do you get the client? Do you use node.client()? (which you should).

On Tuesday, March 22, 2011 at 11:47 PM, Adam Parkin wrote:

I'm trying to setup a unit test for a module I wrote which uses
ElasticSearch to index some documents. Internally to this module a
node & client are created, and the regular Java API calls are used for
creating the index, indexing documents, etc. They all seem to work
correctly when connecting to an already running "global" cluster
running on another machine.

Of course though, I'd like to keep my unit test "contained" to the
machine the test is running on. The docs for ES (http://
www.elasticsearch.org/guide/reference/java-api/client.html) seem to
indicate you do this by setting the Node object to be a "local" node
by the local() method on the NodeBuilder object used to create the
Node.

What I expect to happen is that the node creation code should go from
(in the normal use):

node = nodeBuilder().clusterName(clusterName).node();

to (in the unit test):

node = nodeBuilder().clusterName(clusterName).local(true).node();

And the rest of the code in my module under test should work unchanged
after being supplied with this node. Apparently I am mistaken in this
belief.

If I do the above (append a call to .local(true) to the builder), then
my code which waits for the cluster to go from RED status to non-red
status times out after 50 seconds, and I fail to index.

If I drop the clusterName() & retain the .local() then indexing
completes, but no data seems to be stored (doing a query on the
corresponding client returns no hits).

Am I missing something about the use and purpose of a "local" node?
If this isn't how it's intended to be used, then does anyone have any
tips/suggestions on how to test code which uses ES for indexing? I
suppose I could connect to the global cluster, create a "dummy" test
index, then delete it upon test termination, but this seems dangerous
(what if test fails to finish, or goes haywire?) and creates a
seemingly unnecessary coupling between my global ES cluster and the
machine running a test.

In case it matters, this is running ElasticSearch 0.14.4.

Thanks.


(pzelnip) #13

That was it, thanks!

On Mar 23, 10:20 am, Shay Banon shay.ba...@elasticsearch.com wrote:

When refreshing and you wan to make sure the refresh has executed, you should execute and actionGet it: client.admin().indices().prepareRefresh().execute().actionGet().

Also, when you post a sample, gist it.On Wednesday, March 23, 2011 at 7:11 PM, Adam Parkin wrote:

Here goes:

public void runESTest() {
final String clusterName = "testcluster";
final String indexName = "testindex";
final String typeName = "testtype";

// delete old data/ directory
deleteDataDirectory();

Node node =
nodeBuilder().clusterName(clusterName).local(true).node();
Client client = node.client();

// create the index
try {
waitForReady(client);

client.admin().indices().create(createIndexRequest(indexName))
.actionGet();
} catch (RemoteTransportException e) {
logger.error("Error creating index {}", indexName, e);
System.exit(1);
}

// index a record
String id = "42";
HashMap<String, Object> data = new HashMap<String, Object>();
data.put("firstname", "Douglas");
data.put("lastname", "Adams");

try {
waitForReady(client);
client.prepareIndex(indexName, typeName, id)
.setSource(buildJSON(data)).execute().actionGet();
waitForRefresh(client, indexName);
} catch (ElasticSearchException e) {
logger.error("Failure to index document with ID: {}", id,
e);
} catch (IOException e) {
logger.error("Failure to create JSON to index", e);
}

// now query for the record.
waitForReady(client);
SearchResponse response = client.prepareSearch(indexName)
.setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
.setQuery(fieldQuery("firstname", "Douglas"))
.addFields("firstname",
"lastname").setFrom(0).setSize(60)
.setExplain(true).execute().actionGet();

System.out.println("Showing hits...");
for (SearchHit sh : response.getHits()) {
System.out.println("Showing hit: '"

  • (String) sh.field("firstname").getValue() +
    "'");
    }
    System.out.println("Done Showing hits...");

// close down client & node
waitForReady(client);
client.close();
node.close();
}

private XContentBuilder buildJSON(HashMap<String, Object> items)
throws IOException {

XContentBuilder xcb = jsonBuilder();
xcb.startObject();

for (String key : items.keySet())
xcb.field(key, items.get(key));

xcb.endObject();
logger.info("JSON for record: {}", xcb.string());
return xcb;
}

void waitForReady(Client client) {
// wait 30 seconds for yellow
ClusterHealthRequest req = new ClusterHealthRequest().timeout(
new TimeValue(30,
TimeUnit.SECONDS)).waitForYellowStatus();

ClusterHealthResponse response =
client.admin().cluster().health(req)
.actionGet();

if (response.getStatus() == ClusterHealthStatus.RED) {
logger.error("Cluster failed to reach ready
status...aborting");
throw new RuntimeException(
"ElasticSearch cluster status failed become
ready");
}
}

private static void waitForRefresh(Client client, String
indexName) {
client.admin().indices().refresh(refreshRequest(indexName));
}

This code prints out the Douglas Adams record when I change the
local() call in the NodeBuilder to be false (at least with v0.14.4),
but prints out no records when it is set to true (in both v0.14.4 and
v0.15.2).

Not sure if I'm refreshing the index correctly, so any feedback on the
waitForRefresh() method would be appreciated.

Thanks.

On Mar 23, 9:37 am, Shay Banon shay.ba...@elasticsearch.com wrote:

A test case would be great. Did you refresh after the indexing is done?

On Wednesday, March 23, 2011 at 6:17 PM, Adam Parkin wrote:

If you mean for doing indexing & queries, then yeah I already use the
prepareXXX methods. I took your response to indicate that creation of
the node/client has an equivalent prepareXXX set of methods, which are
not on the site. I think I just misunderstood what you originally
meant. :slight_smile:

Anywho: I tried incorporating Curtis' suggestions (most of which I was
already doing, the exception being doing a refresh on the index after
indexing). No change, had same problem (indexing completes, but no
matches on query).

I also tried switching to v0.15.2, and ran locally, and had no
change. Indexing completes, but the query itself returns no
matches.

My testing code indexes a single record which consisted of two fields:
firstname & lastname, which had values "Douglas" and "Adams", and the
ID for the record was (of course) set to 42.

The query code looks like

SearchResponse response = client.prepareSearch(INDEX_NAME)
.setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
.setQuery(fieldQuery("firstname", "Douglas"))
.addFields("firstname",
"lastname").setFrom(0).setSize(60)
.setExplain(true).execute().actionGet();

And printing the results of the query looks like:

for (SearchHit sh : response.getHits()) {
System.out.println("Showing hit: '"

  • (String) sh.field("firstname").getValue() +
    "'");
    }

Which prints nothing to the screen (though when I ran the same code
without making the node local, I did get "Douglas" printed on the
screen as expected.

I'll try and write up a simple class which exploits the exact behavior
I'm seeing and post back with it.

Thanks.

On Mar 23, 2:17 am, Shay Banon shay.ba...@elasticsearch.com wrote:

Ohh, and the prepareXXX are the only one that are now documented on the site.

On Wednesday, March 23, 2011 at 11:16 AM, Shay Banon wrote:

If you can gist a simple sample of the code that fails, I can give it a go.
On Wednesday, March 23, 2011 at 6:17 AM, Adam Parkin wrote:

I don't wait for Green, but rather yellow.

And yes, the Client is created as:

Client client = node.client();

As for the prepareXXX calls, I'm not familiar with them (they're not
documented on the site so far as I could see), is there a starting
point I should look at to explore them?

I'll try Curtis' suggested ideas along with trying v0.15+ and report
back tomorrow.

Thanks.

On Mar 22, 2:56 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

Running the local node is aimed exactly at unit tests, and actually, elasticsearch has a lot of tests internally that use it.

Maybe the reason why your code hangs when waiting for a cluster health is that you wait for a GREEN health state, where with a single node and replica count > 0 you will never get.

Dropping the cluster name should not change anything. How do you get the client? Do you use node.client()? (which you should).

On Tuesday, March 22, 2011 at 11:47 PM, Adam Parkin wrote:

I'm trying to setup a unit test for a module I wrote which uses
ElasticSearch to index some documents. Internally to this module a
node & client are created, and the regular Java API calls are used for
creating the index, indexing documents, etc. They all seem to work
correctly when connecting to an already running "global" cluster
running on another machine.

Of course though, I'd like to keep my unit test "contained" to the
machine the test is running on. The docs for ES (http://
www.elasticsearch.org/guide/reference/java-api/client.html) seem to
indicate you do this by setting the Node object to be a "local" node
by the local() method on the NodeBuilder object used to create the
Node.

What I expect to happen is that the node creation code should go from
(in the normal use):

node = nodeBuilder().clusterName(clusterName).node();

to (in the unit test):

node = nodeBuilder().clusterName(clusterName).local(true).node();

And the rest of the code in my module under test should work unchanged
after being supplied with this node. Apparently I am mistaken in this
belief.

If I do the above (append a call to .local(true) to the builder), then
my code which waits for the cluster to go from RED status to non-red
status times out after 50 seconds, and I fail to index.

If I drop the clusterName() & retain the .local() then indexing
completes, but no data seems to be stored (doing a query on the
corresponding client returns no hits).

Am I missing something about the use and purpose of a "local" node?
If this isn't how it's intended to be used, then does anyone have any
tips/suggestions on how to test code which uses ES for indexing? I
suppose I could connect to the global cluster, create a "dummy" test
index, then delete it upon test termination, but this seems dangerous
(what if test fails to finish, or goes haywire?) and creates a
seemingly unnecessary coupling between my global ES cluster and the
machine running a test.

In case it matters, this is running ElasticSearch 0.14.4.

Thanks.


(system) #14