ES Cluster stopped working

Elastic Stack Elasticsearch
1

My ES cluster is a 3 node setup cluster running on ES 7.17 , lately its giving alot of issues, So i had to made a single node setup removing the nodes and re-indexed from the application console. Now if I revert back to the cluster from single node what should I do to re-enable my cluster back and how to make sure the new nodes autojoins the cluster?

Also I could see the health of single node as yellow and there are some recoveries not sure how to assign them back
curl -k -X GET "https://localhost:9200/_cat/health"
1730951810 03:56:50 Prod yellow 1 1 9 9 0 0 7 0 - 56.3%

curl -k -X GET "https://localhost:9200/_cat/recovery?pretty"
defect 0 207ms empty_store done n/a n/a 10.X.X.X xxx.xxx.com.au n/a n/a 0 0 0.0% 0 0 0 0.0% 0 0 0 100.0%
build 0 85ms empty_store done n/a n/a 10.X.X.X xxx.xxx.com.au n/a n/a 0 0 0.0% 0 0 0 0.0% 0 0 0 100.0%
testsuite 0 69ms empty_store done n/a n/a 10.X.X.X xxx.xxx.com.au n/a n/a 0 0 0.0% 0 0 0 0.0% 0 0 0 100.0%
release 0 127ms empty_store done n/a n/a 10.X.X.X xxx.xxx.com.au n/a n/a 0 0 0.0% 0 0 0 0.0% 0 0 0 100.0%
requirement 0 168ms empty_store done n/a n/a 10.X.X.X xxx.xxx.com.au n/a n/a 0 0 0.0% 0 0 0 0.0% 0 0 0 100.0%
testcaserun 0 77ms empty_store done n/a n/a 10.X.X.X xxx.xxx.com.au n/a n/a 0 0 0.0% 0 0 0 0.0% 0 0 0 100.0%
testcase 0 98ms empty_store done n/a n/a 10.X.X.X xxx.xxx.com.au n/a n/a 0 0 0.0% 0 0 0 0.0% 0 0 0 100.0%

please help

2

Hi @Techiebee,

A yellow cluster status suggests you have unassigned replicas. Have you followed the troubleshooting steps in the documentation?

Can you check the unassigned shards using the commands documented here, and also the allocation settings?

Hope that helps!

3

From Elastic Search to Elasticsearch

4

Removed elastic-app-search

5

I have started the cluster, and all the indicies have become green, still my cluster isnt working

Suppressed: shaded.qasymphony.org.elasticsearch.client.ResponseException: method [PUT], host [https://xxxx:9200], URI [/requirement/_mapping], status line [HTTP/1.1 400 Bad Request]
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"mapper [externalSystemRequirementSystemField] cannot be changed from type [text] to [keyword]"}],"type":"illegal_argument_exception","reason":"mapper [externalSystemRequirementSystemField] cannot be changed from type [text] to [keyword]"},"status":400}
at shaded.qasymphony.org.elasticsearch.client.RestClient.convertResponse(RestClient.java:283)
at shaded.qasymphony.org.elasticsearch.client.RestClient.performRequest(RestClient.java:261)
at shaded.qasymphony.org.elasticsearch.client.RestClient.performRequest(RestClient.java:235)
at shaded.qasymphony.org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1514)
... 59 more
09-Nov-2024 22:51:09.269 INFO [main] org.apache.catalina.core.ApplicationContext.log Closing Spring root WebApplicationContext

this was intial error due to which I had to switch to single node from cluster and re-index data, but now when i switch back to cluster its the same

6 
[PUT], host [[https://xxxx:9200](https://xxxx:9200/)], URI [/requirement/_mapping], status line [HTTP/1.1 400 Bad Request]
:
"mapper [externalSystemRequirementSystemField] cannot be changed from type [text] to [keyword]"},"status":400}
  • Does index already exist with data in it? Changing the mappings of an existing index with data in it could be one of the reason for this error.
  • externalSystemRequirementSystemField is being modified from [text] to [keyword]
7

I haven't changed any thing, there is already existing data , I have tried re-indexing it , but it still throws the same error when started as cluster, but with single node its not causing issue

8

Could you please confirm when you say single node you are able to start elasticsearch on all 3 nodes ? i mean single node is working only for specific node or all 3 nodes? this can help to understand the issue if the data is up to date on all the 3 elasticnodes or not?

9

Single node is a single server/EC2 running one elasticsearch service, 3 nodes in the above context meaning 3 servers/ec2's as one cluster