Issue with unit testing TransportClient

SD1 · February 13, 2014, 4:04pm

Hi guys,

We are integrating ElasticSearch into our Akka framework. We are using
Akka's Extensions (in case someone here is familiar with Akka) to create
the client like below

public Client getClient() {
return new TransportClient().addTransportAddress(new
InetSocketTransportAddress("localhost", 9300));
}

Now in my unit test, I want to test the method getClient()...

Here are my thoughts to achieve this..

Start up an elastic search test server in my unit test class with a
different cluster name and different port
Pass these modified configuration parameters to getClient()
Test whether the client is created or not..

I am stuck on step 1 itself..I am looking at documentation and various
examples online...Most of them show the ability to create a test Node using
NodeBuilder and getting the client in the test itself or creating a
TransportClient in the test itself..But I need the ability to start the
test elastic server in my unit test and then make my app code to use that
server to get the client..

Any thoughts on how I can achieve this?

Thanks,
SD

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ff2f5a29-59ff-4668-ade2-fbe1091fc926%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

jprante · February 13, 2014, 4:48pm

The NodeBuilder can start an embedded Elasticsearch cluster, this is
exactly what you want for unit tests with a TransportClient.

To see how this works, you can also consult my elasticsearch-support tests
which I stole from the original Elasticearch test code:

https://github.com/jprante/elasticsearch-support/blob/master/src/test/java/org/xbib/elasticsearch/support/AbstractNodeTest.java

Note, is is rather bad practice to issue a "new TransportClient()" each
time you want a client instance. Create just a singleton TransportClient
and pass this instance around. At instantiation, the TransportClient starts
the full discovery and cluster rendevzous, which takes a loooong time.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoG_%2BqrO0dBoOJUDyxY7Wi65702hj8u%2BpajYZ-BRnGQ6JA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

SD1 · February 13, 2014, 5:04pm

Jörg,

Thanks for your response. I see that in your example you create the Node
and get the client in the buildNode() method used by the tests..But my
requirement is different.

I want to be able to start a test Elasticsearch server and make my code in
the app (not the test) use this server to give me the client..

For Example:

Lets say this is how my code is organised

-src package

class ElasticSearchExtension { Client getClient() { return new
TransportClient().addTransportAddress(new
InetSocketTransportAddress(config.get("host"), config.get("port"))); } }
-tests package
class ElasticSearchExtensionTest { testGetClient() { } }

Now in my ElasticSearchExtensionTest, I want to be able to start up a test
elastic server and tell the getClient() in ElasticSearchExtension to use
the test server and give me the client...I don't want to get the client
directly in ElasticSearchExtensionTest.

Also, thanks for your note. But we instantiate the client only once per
module using Akka's Extensions and Bootable implementations.

Thanks,
SD

On Thursday, February 13, 2014 10:48:20 AM UTC-6, Jörg Prante wrote:

The NodeBuilder can start an embedded Elasticsearch cluster, this is
exactly what you want for unit tests with a TransportClient.

To see how this works, you can also consult my elasticsearch-support tests
which I stole from the original Elasticearch test code:

https://github.com/jprante/elasticsearch-support/blob/master/src/test/java/org/xbib/elasticsearch/support/AbstractNodeTest.java

Note, is is rather bad practice to issue a "new TransportClient()" each
time you want a client instance. Create just a singleton TransportClient
and pass this instance around. At instantiation, the TransportClient starts
the full discovery and cluster rendevzous, which takes a loooong time.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9305fe60-9ef7-48e0-a6bd-82d7a788bb6d%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

jprante · February 13, 2014, 6:37pm

I admit the code is confusing and too short, sorry.

What you want is just exactly what I do.

I start a test elasticsearch test cluster by doing this

@BeforeMethod
public void create() throws Exception {
    startNode("1");
    NodesInfoRequest nodesInfoRequest = new

NodesInfoRequest().transport(true);
NodesInfoResponse response =
client("1").admin().cluster().nodesInfo(nodesInfoRequest).actionGet();
InetSocketTransportAddress address =
(InetSocketTransportAddress)response.iterator().next()
.getTransport().getAddress().publishAddress();
PORT = address.address().getPort();
addresses.put("1", address);
....
}

...

protected URI getAddress() {
    return URI.create("es://localhost:" + PORT + "?es.cluster.name=" +

CLUSTER);
}

and now I can tell a TransportClient to use the server. For this I keep a
map of adresses of the server nodes in form of an URI.

Look at this code, which is trivially extending the abstract test code

@Test
public void testBulkClient() {
    final BulkClient es = new BulkClient()
            .newClient(getAddress());
    es.shutdown();
    if (es.hasErrors()) {
        logger.error("error", es.getThrowable());
    }
    assertFalse(es.hasErrors());
}

I use "getAddress()" for the address of the elasticseach test cluster that
is currently running, and the "BulkClient" (which is my custom client
wrapping a TransportClient together with a BulkProcessor) is connecting to
the embedded test cluster with "newClient(...)". In this method I build a
Settings object that contains all the required info for the TransportClient
to connect.

The Setting object is something like

settingsBuilder()
.put("cluster.name", findClusterName(uri))
.put("network.server", false)
.put("node.client", true)
.put("client.transport.sniff", false)
.put("client.transport.ignore_cluster_name", false)
.put("client.transport.ping_timeout", "30s")
.put("client.transport.nodes_sampler_interval", "30s")
.build();

where "findClusterName()" is a routine to get the cluster name from the
parameters of the test.

The @AfterMethod shuts everything down, the "BulkClient" and the test
cluster.

If i want more nodes, I add "startNode("2")", "startNode("3")", to the
@BeforeMethod

Does this help?

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEL2w5kJby8PddkvHRdZNtAima3E1vAaEYOW%3D3mQN2_og%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

spinscale · February 14, 2014, 9:29am

Hey,

you might want to read:

should help you to create tests which have a whole elasticsearch cluster
running in the background.

--Alex

On Thu, Feb 13, 2014 at 7:37 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

I admit the code is confusing and too short, sorry.

What you want is just exactly what I do.

I start a test elasticsearch test cluster by doing this
@BeforeMethod
public void create() throws Exception {
    startNode("1");
    NodesInfoRequest nodesInfoRequest = new
NodesInfoRequest().transport(true);
NodesInfoResponse response =
client("1").admin().cluster().nodesInfo(nodesInfoRequest).actionGet();
InetSocketTransportAddress address =
(InetSocketTransportAddress)response.iterator().next()
.getTransport().getAddress().publishAddress();
PORT = address.address().getPort();
addresses.put("1", address);
....
}

...
protected URI getAddress() {
    return URI.create("es://localhost:" + PORT + "?es.cluster.name="
CLUSTER);
}

and now I can tell a TransportClient to use the server. For this I keep a
map of adresses of the server nodes in form of an URI.

Look at this code, which is trivially extending the abstract test code
@Test
public void testBulkClient() {
    final BulkClient es = new BulkClient()
            .newClient(getAddress());
    es.shutdown();
    if (es.hasErrors()) {
        logger.error("error", es.getThrowable());
    }
    assertFalse(es.hasErrors());
}
I use "getAddress()" for the address of the elasticseach test cluster that
is currently running, and the "BulkClient" (which is my custom client
wrapping a TransportClient together with a BulkProcessor) is connecting to
the embedded test cluster with "newClient(...)". In this method I build a
Settings object that contains all the required info for the TransportClient
to connect.

The Setting object is something like

settingsBuilder()
.put("cluster.name", findClusterName(uri))
.put("network.server", false)
.put("node.client", true)
.put("client.transport.sniff", false)
.put("client.transport.ignore_cluster_name", false)
.put("client.transport.ping_timeout", "30s")
.put("client.transport.nodes_sampler_interval", "30s")
.build();

where "findClusterName()" is a routine to get the cluster name from the
parameters of the test.

The @AfterMethod shuts everything down, the "BulkClient" and the test
cluster.

If i want more nodes, I add "startNode("2")", "startNode("3")", to the
@BeforeMethod

Does this help?

Jörg

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEL2w5kJby8PddkvHRdZNtAima3E1vAaEYOW%3D3mQN2_og%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGCwEM9J7g23LTiDwci6xu5wcTKyTMxpsCancLLdjdKteur6tw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.