Cluster Status Red & Bootstrap Check Failed

[Giuseppe_Merlo]

Hi guys,

i'm using Elasticsearch with a custom application to save data and get analytics.

My Elasticsearch is formed by only a node with so many indexes. This node contains unassigned shards that caused cluster health status red. How can i resolve this issue?

Further, when i run Elasticsearch, i use this command

./elasticsearch -E "discovery.type=single.node"

because if i try to normally start Elasticsearch, it returns this error:

bound or publishing to a non-loopback address, enforcing bootstrap checks
ERROR: [1] bootstrap checks failed
[1]: max number of threads [2048] for user [giuseppe] is too low, increase to at least [4096]

I have modified so many Linux system files but no one works fine.
Can anyone help me?

Thank you a lot in advance!

[programagor (Jiří Bednář)]

Make sure you have the following lines in your /etc/security/limits.conf:

elasticsearch soft memlock unlimited
elasticsearch hard memlock unlimited
elasticsearch  -   nofile  65536
elasticsearch  -   nproc  4096

[Giuseppe_Merlo]

Thank you so much! This works fine now.

Any ideas for cluster red status?

EDIT

I used Cluster Allocation API to see what is going wrong and this is the response:

{
"index": "metrics_timer-2018-07-22",
"shard": 2,
"primary": true,
"current_state": "unassigned",
"unassigned_info": {
"reason": "CLUSTER_RECOVERED",
"at": "2018-08-07T07:37:30.941Z",
"last_allocation_status": "no_valid_shard_copy"
},
"can_allocate": "no_valid_shard_copy",
"allocate_explanation": "cannot allocate because a previous copy of the primary shard existed but can no longer be found on the nodes in the cluster",
"node_allocation_decisions": [
  {
"node_id": "aLwf7SYhRPWog2nbhxzvfQ",
"node_name": "aLwf7SY",
"transport_address": "192.168.1.66:9300",
"node_attributes": {
"ml.machine_memory": "16763133952",
"ml.max_open_jobs": "20",
"ml.enabled": "true"
},
"node_decision": "no",
"store": {
"found": false
}
}
],
}

[programagor (Jiří Bednář)]

Ouch, that looks bad. If you don't mind losing data, then you can try deleting the index [metrics_timer-2018-07-22]. Like this:

DELETE /metrics_timer-2018-07-22

or

curl -X DELETE '192.168.1.66:9200/metrics_timer-2018-07-22'

If you want to keep the data, then you'll need to find at least one replica of shard 2 of that index.
Did you remove a data nodes without having shard replicas?