# Need to start ES instance for test purpose **URL:** \ **Category:** Elasticsearch\ **Created:** [January 17, 2018, 4:09pm UTC](https://discuss.elastic.co/t/need-to-start-es-instance-for-test-purpose/115918 "2018-01-17T16:09:16Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![prakharjain17](https://avatars.discourse-cdn.com/v4/letter/p/d2c977/32.png) [@prakharjain17](https://discuss.elastic.co/u/prakharjain17)\ **Post date:** [January 17, 2018, 4:09pm UTC](https://discuss.elastic.co/t/need-to-start-es-instance-for-test-purpose/115918/1 "2018-01-17T16:09:17Z")
> [@Elasticsearch Maven Plugin using gradle](https://discuss.elastic.co/t/elasticsearch-maven-plugin-using-gradle/115557/8): > > I joined the conversation late and did not fully read your original intention. I simply thought you wanted to use the plugin plugin. That plugin is meant to develop plugins, but it probably can be repurposed to simply run integration tests. Your integration test gradle task would need to depend on the integTestCluster task. I do not think you will be able to use the clients though, so you will need to make sure that the test cluster starts on a port you are expecting. Perhaps there is some … Hi Ivan and others, I would like to discuss what I am doing, and need help regarding it. We have two components - App (Jersey client) and Data (Elasticsearch) We used to have Couchbase as a data component but we are migrating to elasticsearch. In the App component, we serve data from elasticsearch. I need to write tests on the App layer so that we conform to some standards. I was trying to use Elasticsearch test framework (ESIntegTestCase). But it had lot of dependency issues. Basically jar hell was creating problem and logger also. As you guys are not supporting embedded elasticsearch instance. > **[Elasticsearch, the server](https://www.elastic.co/blog/elasticsearch-the-server)** > > How Elasticsearch is evolving into a secure, stable, reliable, predictable solution. And, the documentation is pointing me to use ESIntegTestCase > **[Integration tests | Elasticsearch Guide \[8.11\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/integration-tests.html)** I was in trouble since I wasn't able to integrate it my test due to issues mentioned above. In this issue, nik9000 has mentioned that it is better to spin new instance rather than using ESIntegTestCase, > > > \[This\](https://www.elastic.co/guide/en/elasticsearch/reference/current/integrati…on-tests.html) page points users to inheriting form \`ESIntegTestCase\` to start and stop the cluster. This approach has fallen out of favor with the Elasticsearch project and we don't write new code using it internally. Instead we tend to use http to test against a cluster started by gradle using Elasticsearch's standard startup scripts. This gives us a "more real" test. > > The trouble with this disconnect is that we no longer publish the jars that you'd need to run a fully stood up cluster to maven central. The jars are available in central but inside the zip distribution. The \`ESIntegTestCase\` way of testing looks promising and convenient so users start down that path only to be thwarted by unavailable jars. > > We need to fix this documentation so it expresses our current recommendations. And we'll need to keep it up to date as we come to a conclusion on https://github.com/elastic/elasticsearch/issues/21119. Alecojacoru elasticsearch maven plugin servers this purpose. But I wanted to use it in gradle project. I wasnt able to find any alternative for it. > **[GitHub - alexcojocaru/elasticsearch-maven-plugin: A Maven plugin to run a...](https://github.com/alexcojocaru/elasticsearch-maven-plugin)** > > A Maven plugin to run a single node Elasticsearch cluster during the integration test phase of a build - GitHub - alexcojocaru/elasticsearch-maven-plugin: A Maven plugin to run a single node Elasti... Please help me find solution. Thanks, Prakhar ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [January 17, 2018, 4:23pm UTC](https://discuss.elastic.co/t/need-to-start-es-instance-for-test-purpose/115918/2 "2018-01-17T16:23:43Z")
And you have also this (WIP): > **[dadoonet/testcontainers-java-module-elasticsearch](https://github.com/dadoonet/testcontainers-java-module-elasticsearch)** > > testcontainers-java-module-elasticsearch - Dockerized Elasticsearch container for testing under Testcontainers ---
**Author:** ![prakharjain17](https://avatars.discourse-cdn.com/v4/letter/p/d2c977/32.png) [@prakharjain17](https://discuss.elastic.co/u/prakharjain17)\ **Post date:** [January 17, 2018, 5:05pm UTC](https://discuss.elastic.co/t/need-to-start-es-instance-for-test-purpose/115918/3 "2018-01-17T17:05:32Z")
I am not able to build the above module. Is it present in mvn central. Not able to get it from gradle. Edit: I tried mvn clean install, its failing on tests saying could not find valid docker environment. I need to run this on simple machine. i dont have docker containers to run it on. ---
**Author:** ![prakharjain17](https://avatars.discourse-cdn.com/v4/letter/p/d2c977/32.png) [@prakharjain17](https://discuss.elastic.co/u/prakharjain17)\ **Post date:** [January 18, 2018, 1:22am UTC](https://discuss.elastic.co/t/need-to-start-es-instance-for-test-purpose/115918/4 "2018-01-18T01:22:37Z")
I am trying to figure out how to run it on simple machine. I have a gradle project, and by far the best way I have found it is using esplugin. There I want to use integTestCluster as pre requisite for all my test. (i.e. I want to start elastic search instances before running tests on it) What is the best way to do it? ---
**Author:** ![prakharjain17](https://avatars.discourse-cdn.com/v4/letter/p/d2c977/32.png) [@prakharjain17](https://discuss.elastic.co/u/prakharjain17)\ **Post date:** [January 18, 2018, 10:08am UTC](https://discuss.elastic.co/t/need-to-start-es-instance-for-test-purpose/115918/5 "2018-01-18T10:08:10Z")
Hi, anyone from elasticsearch community, please reply! One more question, we were earlier using embedded elastic search instance for JUnit tests. What approach should we use now? ---
**Author:** ![prakharjain17](https://avatars.discourse-cdn.com/v4/letter/p/d2c977/32.png) [@prakharjain17](https://discuss.elastic.co/u/prakharjain17)\ **Post date:** [January 18, 2018, 2:06pm UTC](https://discuss.elastic.co/t/need-to-start-es-instance-for-test-purpose/115918/6 "2018-01-18T14:06:07Z")
Someone please! _BUMP_ ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [January 18, 2018, 2:21pm UTC](https://discuss.elastic.co/t/need-to-start-es-instance-for-test-purpose/115918/7 "2018-01-18T14:21:32Z")
Read [this](https://discuss.elastic.co/t/about-the-elasticsearch-category/21) and specifically the "Also be patient" part. ---
**Author:** ![prakharjain17](https://avatars.discourse-cdn.com/v4/letter/p/d2c977/32.png) [@prakharjain17](https://discuss.elastic.co/u/prakharjain17)\ **Post date:** [January 18, 2018, 2:52pm UTC](https://discuss.elastic.co/t/need-to-start-es-instance-for-test-purpose/115918/8 "2018-01-18T14:52:19Z")
sorry, but I need some solution urgently. Don't meant to hurt anyone. Thanks, Prakhar ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [January 18, 2018, 3:03pm UTC](https://discuss.elastic.co/t/need-to-start-es-instance-for-test-purpose/115918/9 "2018-01-18T15:03:50Z")
In the link I just pasted you can read: > There are no SLAs on responses to questions posted on this forum, if you require help with an SLA on responses you should look into purchasing a subscription package that includes support with an SLA such as [those offered by Elastic](https://www.elastic.co/subscriptions). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [January 18, 2018, 4:26pm UTC](https://discuss.elastic.co/t/need-to-start-es-instance-for-test-purpose/115918/10 "2018-01-18T16:26:16Z")
At this point, I seriously doubt anyone will answer your question. Elastic support staff do not know anything about internals. Your only hope is for a developer to chime in, but they are rarely here. The code is open source, your best option is to dig in. Given the dependency issues, starting your own instance is the only way to go. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [January 18, 2018, 5:14pm UTC](https://discuss.elastic.co/t/need-to-start-es-instance-for-test-purpose/115918/11 "2018-01-18T17:14:07Z")
@Ivan This is not how support is working. When someone in the support team does not know an answer, they are asking for help from the dev team. And there are SLAs. Here we are all doing our best to answer. Some answers are easy. Some other can take more time. And regarding the problem, I can't answer myself: - I'm not using Gradle apart for plugins in which case the infra available in elasticsearch repo is fine - I'm using most of the time Maven and in that case I used 3 strategies so far which I already shared (I think) with @prakharjain17: - Using ANT Scripts: [https://github.com/dadoonet/fscrawler/blob/master/integration-tests/src/test/ant/integration-tests.xml](https://github.com/dadoonet/fscrawler/blob/master/integration-tests/src/test/ant/integration-tests.xml) - Using elasticsearch maven plugin: [https://github.com/alexcojocaru/elasticsearch-maven-plugin](https://github.com/alexcojocaru/elasticsearch-maven-plugin) - Using Docker based images: [https://github.com/dadoonet/testcontainers-java-module-elasticsearch](https://github.com/dadoonet/testcontainers-java-module-elasticsearch) (which has not been released yet - still WIP so you have to build it locally). Because of my lack of knowledge so far on Gradle, I pinged in another thread @nik9000 and @rjernst who might have time to answer but again there is no SLA on this forum and it could take time for people to answer. ---
**Author:** ![rjernst](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rjernst/32/6363_2.png) [@rjernst](https://discuss.elastic.co/u/rjernst)\ **Post date:** [January 18, 2018, 5:34pm UTC](https://discuss.elastic.co/t/need-to-start-es-instance-for-test-purpose/115918/12 "2018-01-18T17:34:51Z")
As has been pointed out, the gradle `elasticsearch.esplugin` plugin is meant to develop a plugin. However, you can create an `integTest` task in your project which will launch an elasticsearch instance, and run [REST tests](https://github.com/elastic/elasticsearch/tree/master/rest-api-spec). Try applying the `elasticsearch.standalone-rest-test` and `elasticsearch.rest-test` plugins. See projects under the qa dir in elasticsearch for examples. ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [January 18, 2018, 6:12pm UTC](https://discuss.elastic.co/t/need-to-start-es-instance-for-test-purpose/115918/13 "2018-01-18T18:12:05Z")
With the elasticsearch plugins, wouldn't the dependencies get pulled in? At least in the test configuration. Was going to test it out myself. ESIntegTestCase still works for me whenever I do not have dependency conflicts (always Lucene). Let's face it, if it weren't for Ryan, none of these types of questions, at least mine, will ever be answered. 🙂 Thanks Ryan! ---
**Author:** ![prakharjain17](https://avatars.discourse-cdn.com/v4/letter/p/d2c977/32.png) [@prakharjain17](https://discuss.elastic.co/u/prakharjain17)\ **Post date:** [January 19, 2018, 6:05am UTC](https://discuss.elastic.co/t/need-to-start-es-instance-for-test-purpose/115918/14 "2018-01-19T06:05:19Z")
You guys are using ESTestCase, ESIntegTestCase in it. This means, I will get those jar hell issues. ---
**Author:** ![rjernst](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rjernst/32/6363_2.png) [@rjernst](https://discuss.elastic.co/u/rjernst)\ **Post date:** [January 19, 2018, 9:14am UTC](https://discuss.elastic.co/t/need-to-start-es-instance-for-test-purpose/115918/15 "2018-01-19T09:14:45Z")
@Ivan By "elasticsearch plugins" do you mean applying `elasticsearch.esplugin`? If so, then yes, you get an integTest automatically by using that. But you can also create a rest test (ie tasks for starting an ES cluster and test task to run the REST test runner) directly in a project. This is how many projects under the qa directory in elasticsearch work. @prakharjain17 I am not talking about ESTestCase or ESIntegTestCase. What I suggested would allow you to start a real elasticsearch node, and create REST tests against it (this are the high level tests we use to check the elasticsearch apis). You can also create a test in java to connect the cluster. Look at the [client qa test](https://github.com/elastic/elasticsearch/tree/master/qa/smoke-test-client). That particular test uses the transport client, but you could replace the example dependency there with the rest client if that is what you are using. ---
**Author:** ![prakharjain17](https://avatars.discourse-cdn.com/v4/letter/p/d2c977/32.png) [@prakharjain17](https://discuss.elastic.co/u/prakharjain17)\ **Post date:** [January 19, 2018, 9:45am UTC](https://discuss.elastic.co/t/need-to-start-es-instance-for-test-purpose/115918/16 "2018-01-19T09:45:23Z")
> [@rjernst](#): > > client qa test Hi, I am having this gradle file ``` buildscript { repositories { mavenLocal() mavenCentral() jcenter() } dependencies { classpath "org.elasticsearch.gradle:build-tools:6.1.1" } } apply plugin: 'idea' apply plugin: 'java' apply plugin: 'elasticsearch.standalone-rest-test' apply plugin: 'elasticsearch.rest-test' dependencies { // https://mvnrepository.com/artifact/org.elasticsearch.client/elasticsearch-rest-high-level-client compile group: 'org.elasticsearch.client', name: 'elasticsearch-rest-high-level-client', version: '6.1.1' } ``` with following error ``` Cannot add a SourceSet with name 'test' as a SourceSet with that name already exists. ``` ---
**Author:** ![prakharjain17](https://avatars.discourse-cdn.com/v4/letter/p/d2c977/32.png) [@prakharjain17](https://discuss.elastic.co/u/prakharjain17)\ **Post date:** [January 19, 2018, 9:59am UTC](https://discuss.elastic.co/t/need-to-start-es-instance-for-test-purpose/115918/17 "2018-01-19T09:59:07Z")
its getting compiled for this gradle file, but I need to apply the plugin java. Once I have built the gradle file. How to use elasticsearch instance from my code i.e. how to get client from it or else hostname and port. ``` buildscript { repositories { mavenLocal() mavenCentral() jcenter() } dependencies { classpath "org.elasticsearch.gradle:build-tools:6.1.1" } } apply plugin: 'eclipse' apply plugin: 'idea' apply plugin: 'elasticsearch.standalone-rest-test' apply plugin: 'elasticsearch.rest-test' repositories { mavenLocal() mavenCentral() jcenter() } sourceSets { main { java { srcDirs = ['src/java'] } resources { srcDirs = ['src/resources'] } } test { } } dependencies { // https://mvnrepository.com/artifact/org.elasticsearch.client/elasticsearch-rest-high-level-client compile group: 'org.elasticsearch.client', name: 'elasticsearch-rest-high-level-client', version: '6.1.1' } ``` ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [January 22, 2018, 11:32am UTC](https://discuss.elastic.co/t/need-to-start-es-instance-for-test-purpose/115918/18 "2018-01-22T11:32:14Z")
In case it helps, you can read this post from @joel-costigliola : > [@Working EsIntegrationTest example](https://discuss.elastic.co/t/working-esintegrationtest-example/108804): > > Hi, I really struggled to get a working EsIntegrationTest but finally did it, it would have been great to have that out of the box in ES, here's what I did (it should work with maven and gradle): Hope that helps Cheers Which leads to > **[GitHub - joel-costigliola/elastic-search-test: ESIntegTestCase example](https://github.com/joel-costigliola/elastic-search-test)** > > ESIntegTestCase example. Contribute to joel-costigliola/elastic-search-test development by creating an account on GitHub. ---
**Author:** ![prakharjain17](https://avatars.discourse-cdn.com/v4/letter/p/d2c977/32.png) [@prakharjain17](https://discuss.elastic.co/u/prakharjain17)\ **Post date:** [January 22, 2018, 11:51am UTC](https://discuss.elastic.co/t/need-to-start-es-instance-for-test-purpose/115918/19 "2018-01-22T11:51:48Z")
Hi dadoodnet, I have already built this kind of sample project myself. The problem is integrating the test framework in your own project. It creates jar hell problem and logger problem. Anyways, thanks for the reply. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 19, 2018, 11:51am UTC](https://discuss.elastic.co/t/need-to-start-es-instance-for-test-purpose/115918/20 "2018-02-19T11:51:52Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.