Updating opensource project sources from 1->2 of Elasticsearch, integration tests issues :(

Elastic Stack Elasticsearch
1

Hi!
I am upgrading opensource project code from 1.5.1 to 2.3.5 of Elasticsearch.
I've faced with integration tests failing.
I've tried to fix them some ways, and understood a lack of knowledge of elasticsearch kitchen.
There is breaking changes between this versions, and they are huge between v1 and v2 of elasticsearch.
I've used all my google-fu, but there is no useful page explaining actions to upgrade code of plugin tests from 1 to 2.
There is some info about "we changed here this and this", but here is a HUGE lack of explanation with examples, what should be done to integration tests.

So, my question:
ElasticsearchIntegrationTest was refactored to ESIntegTestCase.
There is method org.elasticsearch.test.ESIntegTestCase#httpClient (which is protected, so can be used, maybe, in extending classes?).
If I comment out my own httpClient() in my class and trying to use method ESIntegTestCase#httpClient, there is NullPointerException.
Can't figure out how to fix this issue correctly. :frowning:

So, NPE:

 java.lang.NullPointerException
	at __randomizedtesting.SeedInfo.seed([F85BE188096FD4CF:E74E6E5A4454A696]:0)
	at org.elasticsearch.test.ESIntegTestCase.httpClient(ESIntegTestCase.java:2070) <==
	at com.asquera.elasticsearch.plugins.http.auth.integration.DefaultConfigurationIntegrationTest.testHealthCheck(DefaultConfigurationIntegrationTest.java:45)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)

ESIntegTestCase#httpClient:

protected HttpRequestBuilder httpClient() {
    final NodesInfoResponse nodeInfos = client().admin().cluster().prepareNodesInfo().get();
    final NodeInfo[] nodes = nodeInfos.getNodes();
    assertTrue(nodes.length > 0);
    TransportAddress publishAddress = randomFrom(nodes).getHttp().address().publishAddress(); <== here it fails (line 2070).
    assertEquals(1, publishAddress.uniqueAddressTypeId());
    InetSocketAddress address = ((InetSocketTransportAddress) publishAddress).address();
    return new HttpRequestBuilder(HttpClients.createDefault()).host(NetworkAddress.formatAddress(address.getAddress())).port(address.getPort());
}

Debug says there is node returned from randomFrom(nodes), but it's http field is null.
.getHttp() returns null, and next method called (.address()) against null value, causing NPE.

Maybe there is misconfigured something?
How can I create HTTP-transport port correctly?

Here is also our own httpClient() code from "v1" of plugin:

  public static HttpRequestBuilder httpClient() {
    HttpServerTransport httpServerTransport = internalCluster().getDataNodeInstance(HttpServerTransport.class);
    InetSocketAddress address = ((InetSocketTransportAddress) httpServerTransport.boundAddress().publishAddress()).address();
    return new HttpRequestBuilder(HttpClients.createDefault()).host(address.getHostName()).port(address.getPort());
  }

Any help would be appreciated.

2

I bet http isn't enabled by default for those tests. It usually shouldn't be because ESIntegTestCase extensions usually just use the transport protocol. You might prefer extending your own test classes and just starting Elasticsearch before all your tests.

If you still want to use ESIntegTestCase you should be able to override nodeSettings and do something like

return Settings.builder().put(super.nodeSettings(ordinal)).put("http.enabled", true).build();

That should enable http.

3

Still nothing, it not helped. http field still null.
I will try to dig here, maybe something should be overridden too.

4

Does someone know how to enable http for nodes, created in ESIntegTestCase?
Maybe I am going wrong way trying to still use ESIntegTestCase? :slight_smile:
I've seen REST testing api, so maybe I should refactor this tests?

5

Any progress on this, Антон Мацюк?

6

Sorry, had no time for trying new template-based tests, maybe will do this in this month.

9

After setting http.enabled in NodeSettings, what made it work for me
was accessing httpClient( ) only inside a @Test, or lazily using it as a test class member variable.

class ElasticsearchUtilsIT extends ESIntegTestCase {
httpClient()... // does not work

@Test
public void test() {
httpClient()... /// works.
}

}