Kibana console showing Elastic Search as “Red” status & all shards failed error

Elastic Stack Elasticsearch
1

Hi All,

I face error when i try to run kibana console from my virtual machine, screenshot attached for ref.

Also i checked the logs, i could see below lines which i'm completely not awarekibana_Status_Red .

[2019-06-18T11:19:17,974][DEBUG][o.e.a.s.TransportSearchAction] [elk-denmod-web] All shards failed for phase: [query]
org.elasticsearch.action.NoShardAvailableActionException: null
at org.elasticsearch.action.search.AbstractSearchAsyncAction.start(AbstractSearchAsyncAction.java:122) ~[elasticsearch-5.3.1.jar:5.3.1]
at org.elasticsearch.action.search.TransportSearchAction.executeSearch(TransportSearchAction.java:240) ~[elasticsearch-5.3.1.jar:5.3.1]
at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:146) ~[elasticsearch-5.3.1.jar:5.3.1]
at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:67) ~[elasticsearch-5.3.1.jar:5.3.1]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:170) ~[elasticsearch-5.3.1.jar:5.3.1]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:142) ~[elasticsearch-5.3.1.jar:5.3.1]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:84) ~[elasticsearch-5.3.1.jar:5.3.1]
at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83) ~[elasticsearch-5.3.1.jar:5.3.1]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72) ~[elasticsearch-5.3.1.jar:5.3.1]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:404) ~[elasticsearch-5.3.1.jar:5.3.1]
at org.elasticsearch.client.support.AbstractClient.search(AbstractClient.java:531) ~[elasticsearch-5.3.1.jar:5.3.1]
at org.elasticsearch.rest.action.search.RestSearchAction.lambda$prepareRequest$1(RestSearchAction.java:73) ~[elasticsearch-5.3.1.jar:5.3.1]
at org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:80) [elasticsearch-5.3.1.jar:5.3.1]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:260) [elasticsearch-5.3.1.jar:5.3.1]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:199) [elasticsearch-5.3.1.jar:5.3.1]
at org.elasticsearch.http.netty4.Netty4HttpServerTransport.dispatchRequest(Netty4HttpServerTransport.java:505) [transport-netty4-5.3.1.jar:5.3.1]
at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:72) [transport-netty4-5.3.1.jar:5.3.1]
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349) [netty-transport-4.1.7.Final.jar:4.1.7.Final]

Please let me know , what i need to do to resolve this issue.

Thanks in advance.

2

This is the health output got from the url : http://localhost:9200/_cluster/health?pretty

{
"cluster_name" : "elk-denmod",
"status" : "yellow",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 0,
"active_primary_shards" : 0,
"active_shards" : 0,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 24,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 0.0
}

3

Hi Joseph,
try this:

GET /_cluster/allocation/explain

This can be helpful: https://www.elastic.co/blog/red-elasticsearch-cluster-panic-no-longer

1 Like
4

As you can see from the health check, your cluster has no data nodes.

You will either have to set node.data: true on the one node you have or add a dedicated data node to your cluster. Otherwise Elasticsearch will have nowhere to assign the 24 unassigned shards you currently have in the cluster.

1 Like
5

where do i need to run this line of code ?
as im new to this, im not aware.
Please help

6

Even if i change the node status to true, still it shows as red.

7

Did you restart Elasticsearch after setting node.data: true in the config/elasticsearch.yml file?

If you did, what does health tell you now? Is "number_of_data_nodes" 1 or still 0?

If you now have a data node in the cluster, the red health must be due to one or more red indices.

8

For example you can run command from:

1) Kibana
You can run command in Kibana - Dev Tools - Console.

2) Curl
You can use curl utility in command line

curl -XGET localhost:9200
9

Edit your elasticsearch.yml file located in the conf repo of Elasticsearch. Set node.data: true and restart your Elasticsearch instance

1 Like
10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.