# Migrating from Elastic 2 to 5

**URL:** <https://discuss.elastic.co/t/migrating-from-elastic-2-to-5/154353>\
**Category:** Elasticsearch\
**Created:** [October 28, 2018, 6:51pm UTC](https://discuss.elastic.co/t/migrating-from-elastic-2-to-5/154353 "2018-10-28T18:51:59Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![praveenoma](https://avatars.discourse-cdn.com/v4/letter/p/9de053/32.png) [@praveenoma](https://discuss.elastic.co/u/praveenoma)\
**Post date:** [October 28, 2018, 6:52pm UTC](https://discuss.elastic.co/t/migrating-from-elastic-2-to-5/154353/1 "2018-10-28T18:52:00Z")

</div>

1. We are migrating from Spring Boot 1.x (comes with Elastic 2) to  
Spring Boot 2.x (comes with Elastic 5)  
In Spring boot 1 (i.e Elastic 2), we've used the NodeBuilder api to create in memory nodes on demand.

2. With SpringBoot 2(i.e Elastic 5), Nodebuilder is not available, we tried using below code:

```auto
   File tmpDir = File.createTempFile("elastic",Long.toString(System.nanoTime()));
    Settings settings = Settings.builder()
            .put("http.enabled", true)
              .put("index.number_of_shards", "1")
              .put("path.data", new File(tmpDir, "data").getAbsolutePath())
              .put("path.logs", new File(tmpDir, "logs").getAbsolutePath())
              .put("path.work", new File(tmpDir, "work").getAbsolutePath()) 
              .put("transport.tcp.port", "9300")
              .put("network.host", "localhost")              
              .put("node.data", "true")
              .put("node.master", "true")

             .put("path.home", tmpDir)
            .put("transport.type", "local").build();
    return new ElasticsearchTemplate(new Node(settings).start().client());

```

1. When we try to access the ElasticsearchRepository, it is giving below exception:

```auto
NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{6gkUXEX6Rsyowd928D0k_A}{localhost}{127.0.0.1:9200}]]
        at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:347)
        at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:245)
        at org.elasticsearch.client.transport.TransportProxyClient.execute(TransportProxyClient.java:59)
        at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:363)
        at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:408)
        at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:80)
        at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:54)
        at org.springframework.data.elasticsearch.core.ElasticsearchTemplate.doCount(ElasticsearchTemplate.java:495)
        at org.springframework.data.elasticsearch.core.ElasticsearchTemplate.count(ElasticsearchTemplate.java:473)
        at org.springframework.data.elasticsearch.repository.support.AbstractElasticsearchRepository.count(AbstractElasticsearchRepository.java:150)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:483)
        at org.springframework.data.repository.core.support.RepositoryComposition$RepositoryFragments.invoke(RepositoryComposition.java:377)
        at org.springframework.data.repository.core.support.RepositoryComposition.invoke(RepositoryComposition.java:200)
        at org.springframework.data.repository.core.support.RepositoryFactorySupport$ImplementationMethodExecutionInterceptor.invoke(RepositoryFactorySupport.java:641)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
        at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.doInvoke(RepositoryFactorySupport.java:605)
        at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.invoke(RepositoryFactorySupport.java:590)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
        at org.springframework.data.projection.DefaultMethodInvokingMethodInterceptor.invoke(DefaultMethodInvokingMethodInterceptor.java:59)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
        at org.springframework.data.repository.core.support.SurroundingTransactionDetectorMethodInterceptor.invoke(SurroundingTransactionDetectorMethodInterceptor.java:61)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
        at com.sun.proxy.$Proxy139.count(Unknown Source)
        at com.werner.ecp.elastic.service.carrier.ElasticCarrierServiceImpl.getCarrierCount(ElasticCarrierServiceImpl.java:58)
        at com.werner.ecp.elastic.service.carrier.ElasticCarrierServiceImpl$FastClassBySpringCGLIB$e96152a4.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

```

1. Any thoughts on what am missing? appreciate your help!

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [October 29, 2018, 11:43am UTC](https://discuss.elastic.co/t/migrating-from-elastic-2-to-5/154353/2 "2018-10-29T11:43:38Z")

</div>

Embedding Elasticsearch into your application like this isn't really a recognised way of running Elasticsearch any more. [This blog post from 2016](https://www.elastic.co/blog/elasticsearch-the-server) describes the reasoning for this decision.

It may be possible to get it to work like this still, but it's not something we cover in any tests so it might be fragile and difficult to debug. If at all possible, you should run Elasticsearch as a standalone process instead.

---

<div class="post-metadata">

**Author:** ![praveenoma](https://avatars.discourse-cdn.com/v4/letter/p/9de053/32.png) [@praveenoma](https://discuss.elastic.co/u/praveenoma)\
**Post date:** [November 15, 2018, 7:20pm UTC](https://discuss.elastic.co/t/migrating-from-elastic-2-to-5/154353/3 "2018-11-15T19:20:43Z")

</div>

Thanks Dave for your feedback.

If we move out of 'embedded Elastic server within the application' and move into external standalone elastic search server, how do we set up clustering?  
In production, our application runs on 4 servers . Does that mean we need to run Elastic search instance on these 4 DMZ servers? If yes, how do we make sure the 4 Elastic search instance are running in a clustered fashion?  
Any feedback/pointers will be highly appreciated!

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [November 16, 2018, 9:21am UTC](https://discuss.elastic.co/t/migrating-from-elastic-2-to-5/154353/4 "2018-11-16T09:21:57Z")

</div>

> [@praveenoma](#):
>
> how do we set up clustering

The setup in your original post did not look like a clustered installation, so perhaps you do not need this? The reference manual covers the configuration needed, particularly this page:

[https://www.elastic.co/guide/en/elasticsearch/reference/current/discovery-settings.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/discovery-settings.html)

You do not necessarily need to run an Elasticsearch node on each of your servers, because you can communicate with Elasticsearch over HTTP.

---

<div class="post-metadata">

**Author:** ![praveenoma](https://avatars.discourse-cdn.com/v4/letter/p/9de053/32.png) [@praveenoma](https://discuss.elastic.co/u/praveenoma)\
**Post date:** [November 16, 2018, 6:25pm UTC](https://discuss.elastic.co/t/migrating-from-elastic-2-to-5/154353/5 "2018-11-16T18:25:59Z")

</div>

This is how our current set up looks like

 ![Current%20Embedded%20Elastic%20setup](https://us1.discourse-cdn.com/elastic/original/3X/8/9/893f62bdb1279cf4bc8f4f43a2a39576c36177ff.jpeg)

If am running only one external Elastic server, how do we accomplish the load balance and failover piece?

Do we have a way to get on a phone call or meeting?

Thanks Much.

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [November 16, 2018, 7:59pm UTC](https://discuss.elastic.co/t/migrating-from-elastic-2-to-5/154353/6 "2018-11-16T19:59:52Z")

</div>

> [@praveenoma](#):
>
> If am running only one external Elastic server, how do we accomplish the load balance and failover piece?

It was not clear from your OP that this was your goal. A resilient Elasticsearch cluster needs at least 3 master-eligible nodes and at least two data nodes (some nodes may be both, so 3 nodes in total).

> [@praveenoma](#):
>
> Do we have a way to get on a phone call or meeting?

Although this community forum isn't really the right place to push commercial things, there's no real-time community support offering so the most helpful way I can answer this is to point out that Elastic can help with support and consulting, and this subject matter is also covered in training: [https://www.elastic.co/services](https://www.elastic.co/services)

---

<div class="post-metadata">

**Author:** ![praveenoma](https://avatars.discourse-cdn.com/v4/letter/p/9de053/32.png) [@praveenoma](https://discuss.elastic.co/u/praveenoma)\
**Post date:** [November 16, 2018, 10:16pm UTC](https://discuss.elastic.co/t/migrating-from-elastic-2-to-5/154353/7 "2018-11-16T22:16:05Z")

</div>

Sorry for the confusion.

I tried some clustering on my local machine by making below edits to the shipped elasticSearch.yml:

**Node 1;**  
cluster.name: my-elastic-cluster  
node.name: my-master-node  
node.master: true  
node.data: false  
network.host: 1.X.Y.Z  
Discovery.zen.ping.unicast.hosts: ["1.X.Y.Z:9300", "127.0.0.1:9300"]

**Node 2;**  
cluster.name: my-elastic-cluster  
node.name: my-data-node  
node.master: false  
node.data: true  
network.host: 127.0.0.1  
Discovery.zen.ping.unicast.hosts: ["1.X.Y.Z:9300", "127.0.0.1:9300"]

Although both the nodes come up fine.

1. When I try to --\> [http://1.X.Y.Z:9200/\_cluster/health?pretty](http://1.X.Y.Z:9200/_cluster/health?pretty)  
I get  
{  
"cluster\_name" : "praveen-elastic-cluster",  
**"status" : "red"** ,  
"timed\_out" : false,  
"number\_of\_nodes" : 2,  
"number\_of\_data\_nodes" : 1,  
"active\_primary\_shards" : 0,  
"active\_shards" : 0,  
"relocating\_shards" : 0,  
"initializing\_shards" : 0,  
**"unassigned\_shards" : 10,**  
"delayed\_unassigned\_shards" : 0,  
"number\_of\_pending\_tasks" : 0,  
"number\_of\_in\_flight\_fetch" : 0,  
"task\_max\_waiting\_in\_queue\_millis" : 0,  
"active\_shards\_percent\_as\_number" : 0.0  
}

2. Also, when I try to index a document via postman client, I get this:

"error": {"root\_cause": [ {  
"type": "unavailable\_shards\_exception",  
"reason": "[accounts][3] primary shard is not active..... }

```
 Guessing if 1 is fixed, 2 will be fixed.

```

**Question:**

1. Any thoughts around what changes do I need to make on my local so that the my local cluster status is green and doesn't have any unassigned shards?

2. Also, would you have a sample elasticsearch.yml files for the above mentioned 3 node cluster? where each nodes are running on different machines?

Thanks a bunch for your continuous help!

---

<div class="post-metadata">

**Author:** ![praveenoma](https://avatars.discourse-cdn.com/v4/letter/p/9de053/32.png) [@praveenoma](https://discuss.elastic.co/u/praveenoma)\
**Post date:** [November 17, 2018, 5:15am UTC](https://discuss.elastic.co/t/migrating-from-elastic-2-to-5/154353/8 "2018-11-17T05:15:30Z")

</div>

I cleaned up some folders under elastic directory and am able to run elastic nodes on my local in a clustered fashion. Thanks Dave for all your help!

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [November 17, 2018, 11:17am UTC](https://discuss.elastic.co/t/migrating-from-elastic-2-to-5/154353/9 "2018-11-17T11:17:05Z")

</div>

You're welcome 🙂

For future reference, if you have an unassigned shard, the [allocation explain API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-allocation-explain.html) is the first thing you should look at, as it will try and explain why things couldn't be allocated.

---

<div class="post-metadata">

**Author:** ![praveenoma](https://avatars.discourse-cdn.com/v4/letter/p/9de053/32.png) [@praveenoma](https://discuss.elastic.co/u/praveenoma)\
**Post date:** [November 19, 2018, 3:03am UTC](https://discuss.elastic.co/t/migrating-from-elastic-2-to-5/154353/10 "2018-11-19T03:03:21Z")

</div>

Dave - Now that am able to make some progress on how to set up ES cluster locally, I have my next set of questions for you! 🙂

**Our current Elastic implementation:**  
Ours is a SpringBoot (SprinBoot 1.5) application. We have used Spring Boot API to implement  
_Embedded Elastic_ (i.e its an in-memory elastic instance within the app) and we use _Node client_ to connect to our embedded elastic instance.

**What we like to get to:**

- We are looking to migrate from _Embedded Elastic_ to an _External Elastic server_.
- Migrate from using _Node Client_ and to using a _Rest Client_

**Questions For you:**

**1)** Any pointers you can provide on how we go about migrating our application code from NodeClient to using Rest Client?  
**2)** They talk about Low Level Rest Client and High Level Rest Client. Which one would you recommend us to use? High Level Rest client is available from what version of Elastic?

Thanks Much for your feedback!

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [November 19, 2018, 6:33am UTC](https://discuss.elastic.co/t/migrating-from-elastic-2-to-5/154353/11 "2018-11-19T06:33:05Z")

</div>

> [@praveenoma](#):
>
> **1)** Any pointers you can provide on how we go about migrating our application code from NodeClient to using Rest Client?

Not really, I've never done such a migration myself. The client APIs are designed to be quite similar in shape. Are you stuck on something specific?

> [@praveenoma](#):
>
> **2)** They talk about Low Level Rest Client and High Level Rest Client. Which one would you recommend us to use? High Level Rest client is available from what version of Elastic?

You should use the High-level REST client as much as possible, but it does not cover 100% of the APIs so you might need to use the Low-level REST client from time to time. You can use both clients in one application (in fact the high-level one is implemented on top of the low-level one).

---

<div class="post-metadata">

**Author:** ![praveenoma](https://avatars.discourse-cdn.com/v4/letter/p/9de053/32.png) [@praveenoma](https://discuss.elastic.co/u/praveenoma)\
**Post date:** [November 19, 2018, 5:56pm UTC](https://discuss.elastic.co/t/migrating-from-elastic-2-to-5/154353/12 "2018-11-19T17:56:30Z")

</div>

Any sample code or links you can share around High and Low level rest client?

---

<div class="post-metadata">

**Author:** ![praveenoma](https://avatars.discourse-cdn.com/v4/letter/p/9de053/32.png) [@praveenoma](https://discuss.elastic.co/u/praveenoma)\
**Post date:** [November 19, 2018, 6:04pm UTC](https://discuss.elastic.co/t/migrating-from-elastic-2-to-5/154353/13 "2018-11-19T18:04:36Z")

</div>

"Are you stuck on something specific?" --\> just looking to see how to get started/proceed with this migration in general..

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 17, 2018, 6:04pm UTC](https://discuss.elastic.co/t/migrating-from-elastic-2-to-5/154353/14 "2018-12-17T18:04:44Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.