Moving license from old cluster nodes to new ones

Elastic Stack Elasticsearch
1

We have a 3 node elasticsearch cluster. (node-1, node-2, node-3)
We have a need to introduce 3 new nodes and fail out the original 3.
I have built nodes 4, 5 and 6 and successfully added them to cluster.
Once the cluster settled down i dropped out node1 and let things settle.
I then dropped node2 and let things settle.

Finally when i do the same with node 3 the whole cluster stops and complains about a license issue.

How do I make sure that new nodes actually use the license we have?

Nick

2

Hi @Nick.Evans Welcome to the community!

You have a license / subscription you should open a support ticket.

3

@Nick.Evans are you trying to rotate the Elasticsearch hosts? If yes, the better way to do is as:

  1. add a new node and let it join the cluster

  2. exclude the IP of the ES node which you want to rotate/ replace

  3. let the shards move out of the excluded node till shard count is 0 (can also be 1 some cases but you can confirm from GET _cat/shards?v command if anything is being moved out of the excluded node).

  4. retire the old host and repeat this process.

4

@stephenb I have requested access to be able to log support tickets but while that happens thought I would try asking the community.

@Ayush_Mathur this is effectively what I have done but I added all of the new nodes first. Excluded all of the old nodes then started dropping the old nodes 1 by 1, letting any shard movement finishing before dropping the next one. All was fine until it came to dropping the last old nodes which is when the licence error shows up.

Nick

5

What Version are you on?

What level of license?

Did you change / add any node roles?

Do you have access to the license file that you originally received?

Exactly what error and where are you seeing it?

What is the output of

GET /_license

6

Hi @stephenb

Thanks for the quick response. To answer your questions

Version: 8.5.3
license level: enterprise
access to license file: yes and i even tried re-uploading that to the cluster before dropping the last node out but it didn't make a difference
Exact error: I will have have to upload this when i get to work on monday but Kibana cluster status page shows a toast window in the bottom right saying something along the lines of not being able to evaluate the current license. Going to another pages shows an error in JSON which i will have to capture on Monday
License output:

{
  "license": {
    "status": "active",
    "uid": "e....<Removed>....7",
    "type": "enterprise",
    "issue_date": "2022-07-29T00:00:00.000Z",
    "issue_date_in_millis": 1659052800000,
    "expiry_date": "2023-07-31T23:59:59.999Z",
    "expiry_date_in_millis": 1690847999999,
    "max_nodes": null,
    "max_resource_units": 1,
    "issued_to": "MCRI",
    "issuer": "API",
    "start_date_in_millis": 1659312000000
  }
}
7

Kibana typically connects to a single node.
Check to make sure that the node you're Kibana is pointing at is now a new node.
Restart Kibana
completely clear browser cache
Restart browser
Try logging back in

8

Thanks @stephenb

I had thought my kibana config to talk to a round robin entry for my elastic servers.
As per you advise i checked the configuration and found that kibana is configured to talk specifically to the old servers... After updating this config, restarting kibana and then trying again to remove the final elastic server i am no longer experiencing the licensing problem.

Thanks again for you help.

Nick

1 Like
9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.