ESSingleNodeTestCase / unit tests / official recommendations?

Elastic Stack Elasticsearch
1

I asked this question regarding unit tests a couple of weeks ago: Best practices for automated tests

Now I would like to come back to the ESSingleNodeTestCase-class.

I managed to get it to start an elastic server with the steps mentioned at the end of this post in case anybody is interested.

Now I realized that the Client provided by the ESSingleNodeTestCase is from the org.elasticsearch.client.internal-package. However the regular client that we are using with API version 8 is coming from the co.elastic.clients.elasticsearch-package. In other words, they are not compatible, even though they have similar interfaces. This means I can not use this base class to e.g. test my queries created using the fluent API from v8. And this is exactly what I would like to do.

So here are a couple of questions:

  • Is ESSingleNodeTestCase actually something that should be used by external developers or is it only meant for internal purposes?
  • Are there any plans to migrate ESSingleNodeTestCase to version 8?
  • Is there any (other?) way that an elastic instance can be emulated so it can used in unit tests that can be executed everywhere (locally, build server, etc.)?

Maybe some background on why I believe this would be so powerful:

When writing code that uses an SQL-DB we tend to use h2database for unit tests. This has a lot of benefits:

  • Makes development easier as we wouldn't even need a local DB.
  • Makes unit tests more meaningful as we need less mocking/faking.
  • Or in other words: I guess we should be able to run our unit tests in an as close to a real environment as possible and do it as early as possible ("Shift-left").
  • You could argue that writing tests that access a database-like thing (be it an SQL-DB or elastic, etc.) is a no-go as this should rather be done in an integration test. I understand that. But on the other hand I believe you could consider the DB an integral part of the application, that accesses it. Especially queries and co. are very tightly coupled to the database. Hence they should be tested in unit tests.

I'm wondering if there are any official recommendations from elastic on how to deal with all of this?

I found this: elasticsearch-8.5.3.tar.gz: .../TESTING.asciidoc | Fossies (see chapter " How to write good tests?"). However I assume this rather meant for elastic-internal tests.

As mentioned above, here are the steps required to make ESSingleNodeTestCase work:

It is very important to use junit4 (not 5!), as the base class uses annotations from junit4 (at least at the time of writing).

Add an own policy file (java_policy_tests.policy):

grant {
  permission java.lang.reflect.ReflectPermission "suppressAccessChecks";
  permission org.elasticsearch.secure_sm.ThreadPermission "modifyArbitraryThreadGroup";
};

Make sure tests are started using that file:

<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-surefire-plugin</artifactId>
	<configuration>
	   <argLine>
		  -Djava.security.policy==./java_policy_tests.policy
	   </argLine>
	</configuration>
</plugin>

Then I simply inherited from ESSingleNodeTestCase and the server was started when executing the tests. By using client() provided from the base class

2

Yes pretty much just internal. External clients use HTTP, but the Client you get in this test harness is the one that can be used from within a node and works directly with the node-to-node transport protocol. They have somewhat similar APIs today but definitely won't be compatible. That's deliberate, and not considered a bug.

There are also many "internal" tests which work by starting a real Elasticsearch cluster and talking to it over HTTP. I think you should do that.

3

In fact there's even a new test framework being developed to make this easier - see e.g. Migrate core rest tests with security to new testing framework by mark-vieira · Pull Request #92575 · elastic/elasticsearch · GitHub

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.