Elastic search status is red

Elastic Stack Elasticsearch
1

My Elastic search status is red in colour. How can i trouble shoot and fix this. There is no issue accessing and using it. But I was a bit concerned .

By the way I am running it on single server. There is no cluster setup.

image
image.png709×154 7.78 KB

2

You have at least one index missing at least one primary shard.

Check:

GET _cat/health?v
GET _cat/indices?v
3

I have deleted some of the red indices still I keep getting this filebeat indices.

GET _cat/health?v

epoch      timestamp cluster       status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1555133477 05:31:17  elasticsearch red             1         1    147 147    0    0       10             0                  -                 93.6%

GET _cat/indices?v
Just added red ones here.

health status index                           uuid                   pri rep docs.count docs.deleted store.size pri.store.size
red    open   filebeat-6.7.0-2019.04.05       -9qSlGtvS2uSa65VCtKgwg   3   0       1314            0    627.5kb        627.5kb
red    open   filebeat-6.6.2-2019.04.02       evSL5jAyTCOIfv4SxMXnbQ   3   0        147            0    289.2kb        289.2kb
red    open   filebeat-6.7.0-2019.04.08       nOvZYqhATB2cwewlQOkFyQ   3   0       1136            0    551.2kb        551.2kb
red    open   filebeat-6.6.2-2019.04.05       VzGi8TksTTCwDtOQtYQ4NA   3   0        139            0    212.5kb        212.5kb
red    open   filebeat-6.6.2-2019.04.04       vGPDS5KFS82gzdsAGjAnpQ   3   0        103            0    250.4kb        250.4kb
red    open   filebeat-6.6.2-2019.04.01       lAenK5x3S6K34KivTiQrNA   3   0         54            0    165.4kb        165.4kb
red    open   filebeat-6.7.0-2019.04.04       o58QYOQnROqqxhat1FBnrQ   3   0       1307            0    569.8kb        569.8kb
red    open   filebeat-6.6.2-2019.04.03       lgBzoNz2S6Sed8IGAZ7tGQ   3   0        157            0    273.9kb        273.9kb
red    open   filebeat-6.6.2-2019.04.07       djEM-YxcSay7h5f2Zw06zA   3   0        115            0    189.8kb        189.8kb
red    open   filebeat-6.7.0-2019.04.01       3BXQq2-EQAiVXse62aQvwQ   3   0        431            0      426kb          426kb
4

You probably have too many shards per node.

May I suggest you look at the following resources about sizing:

https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing

And https://www.elastic.co/webinars/using-rally-to-get-your-elasticsearch-cluster-size-right

5

HI @dadoonet

I have seen this link earlier. It only refers to a cluster in elastic search. In my case I do not have a cluster I'm running ES on a single node. Currently I have a default indices per day as default. To reduce the shard size probably I need to move to weekly index. Not sure if that helps.

I have also added lifecycle management to delete indices after 10 days for metric data. But looks like its not working. I have added indices to these life cycle policies but I shows a 0 linked indices.

image
image.png1349×289 13 KB

I am also planning to use rollover indices. Haven't tried yet.

6

You do have a cluster. A cluster with onde node is a cluster.

It will. And also change the number of shards to 1 will be a great win.

7

@dadoonet

Can you please tell me where I can set this limitation. I was looking at this page but its not clear should it be done in elasticsearch.yml or run from dev tool.

https://www.elastic.co/guide/en/elasticsearch/reference/current/allocation-total-shards.html

8

It's an index setting. See https://www.elastic.co/guide/en/elasticsearch/reference/7.0/indices-create-index.html#create-index-settings

9

I was looking at one of my indices which is red in status. I can see number of active shards as 1. Does it mean it has only one shard?

image
image.png905×571 37.4 KB

What does the primaries indicate? I see this in common for indices which are red in color.

image
image.png848×535 30 KB

10

when I try to run this code snippet I get error. So where can i modify the index settings.

PUT filebeat-6.7.0-2019.04.01
{
    "settings" : {
        "number_of_shards" : 1,
        "number_of_replicas" : 0
    }
}


{
  "error": {
    "root_cause": [
      {
        "type": "resource_already_exists_exception",
        "reason": "index [filebeat-6.7.0-2019.04.01/3BXQq2-EQAiVXse62aQvwQ] already exists",
        "index_uuid": "3BXQq2-EQAiVXse62aQvwQ",
        "index": "filebeat-6.7.0-2019.04.01"
      }
    ],
    "type": "resource_already_exists_exception",
    "reason": "index [filebeat-6.7.0-2019.04.01/3BXQq2-EQAiVXse62aQvwQ] already exists",
    "index_uuid": "3BXQq2-EQAiVXse62aQvwQ",
    "index": "filebeat-6.7.0-2019.04.01"
  },
  "status": 400
}
11

You can't modify the number of primary shards for an existing index.
Because you have only one node, no replicas, indices in RED are not complete and functional anymore. You should delete them.
For the existing green/yellow ones you can use the shrink API to reduce the number of shards.

12

Thanks for your reply. Is there a setting in Elastic search or Filebeat where I can define these settings before an index is created. Since I have daily indices being created I can limit the number of shards for those indices in future.

13

This is what index templates are made for.
And filebeat has such a setting as well.

14

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.