Remove a node From a cluster

Elastic Stack Elasticsearch
1

Hi,
I would like to remove a node from my cluster without losing data.

How can I achieve this.

Many Thanks.

2

Use allocation filtering by IP/node name to remove shards from the host, then shut it down.

3

thanks for your replay,
in fact the health of the cluster is Red.. Can I do this without losing data?
I will replace the node with an other node after I exclude it from the cluster.

What do you think please!
thanks

4

It's not guaranteed. You would need to fix that first.

5

Please if you have any idea could you help me! How can I fix the health of the cluster I have 7 nodes...can I reduce the number of shards ?? Or is it a problem of performence?

Many thanks.

6

If the cluster is RED it means that you are yet missing some primary shards which means that you have may be lost some data.

Why the cluster is RED? What happened?
Did you set number of replicas to 0 at some point?

7

Hi
Thank You for your replay,

I kept the DEFAULT number of shards: 5 primary shards and 1 replicas

8

First, you need identify the cause of RED STATE. You can look into `/_cluster/state' output for problematic shards and fix them.

9

the cause is : unassigned shards
it can be a problem of memory space?

10

You can check out the cluster allocation explain API for the cause.

https://www.elastic.co/guide/en/elasticsearch/reference/5.5/cluster-allocation-explain.html

11

Many thanks for your replay

the reason is:"unassigned","unassigned_info":{"reason":"CLUSTER_RECOVERED"

please how can I solve this issue.

12

What is the value against allocate_explanation key for cluster allocation explain API?

13

"can_allocate": "throttled",
"allocate_explanation": "allocation temporarily throttled",
"node_allocation_decisions": [
{
"node_id": "ID",
"node_name": "name",
"transport_address": "@:9300",
"node_attributes": {
"ml.machine_memory": "4294365184",
"ml.max_open_jobs": "20",
"ml.enabled": "true"
},

14

How many shards per node do you have? I believe you have larger number of shards / node.

A similar thread can be found here.

15

Hi,
Many thanks for helping me

the output of GET /_cluster/health?pretty=true is the following:

 {
  "cluster_name": "SIEM-cluster",
  "status": "red",
  "timed_out": false,
  "number_of_nodes": 7,
  "number_of_data_nodes": 6,
  "active_primary_shards": 1616,
  "active_shards": 1616,
  "relocating_shards": 0,
  "initializing_shards": 0,
  "unassigned_shards": 1692,
  "delayed_unassigned_shards": 0,
  "number_of_pending_tasks": 0,
  "number_of_in_flight_fetch": 0,
  "task_max_waiting_in_queue_millis": 0,
  "active_shards_percent_as_number": 48.851269649334945
}
16

Based on the above output, I see 550 shard allocations to a single ES node (1616 + 1692)/6. How much heap space you have allocated? What is the disk size and is there ample disk space available?

17

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.