Alert: Large shard size

[Michele_De_Benedet]

Hi,

I am using Logstash to feed an index. Current index size is 60GB and I'm expecting it to grow to at most 100GB, so I set it to use 6 shards. Elasticsearch is ran on a single node.

The status of the index in Kibana is as follows:
Status Health Green
Alerts 1
Total 60.9 GB
Primaries 60.9 GB
Documents 63.5m
Total shards 6
Unassigned shards 0

And the red alert says:
The following index: [index_name] has a large shard size of: 65.35GB at October 10, 2022 9:15 AM CEST

But how can it be? I am using 6 shards, each having a 10GB size. How can I fix it?

Logstash output configuration:

output {
  # stdout { }
  elasticsearch {
    hosts => ...
    user => ...
    password => ...
    index => "index_name"
    doc_as_upsert => true
    action => "update"
    document_id => ...
    template => "/usr/share/logstash/template/index_name.json"
    template_name => "index_name"
  }
}

Template configuration:

{
  "index_patterns": ["index_name"],
  "settings": {
    "index": {
      "number_of_shards": 6,
      "number_of_replicas": 0
    }
  }
}

Get shards

index_name                 4 p STARTED    10587624  10.4gb * elasticsearch
index_name                 1 p STARTED    10585296   9.9gb * elasticsearch
index_name                 2 p STARTED    10585089   9.9gb * elasticsearch
index_name                 5 p STARTED    10584801   9.9gb * elasticsearch
index_name                 3 p STARTED    10588147  10.1gb * elasticsearch
index_name                 0 p STARTED    10590486  10.4gb * elasticsearch

Shard status (GET _cluster/health/?level=shards):

  "index_name" : {
      "status" : "green",
      "number_of_shards" : 6,
      "number_of_replicas" : 0,
      "active_primary_shards" : 6,
      "active_shards" : 6,
      "relocating_shards" : 0,
      "initializing_shards" : 0,
      "unassigned_shards" : 0,
      "shards" : {
        "0" : {
          "status" : "green",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 0
        },
        "1" : {
          "status" : "green",
          "primary_active" : true,
          "active_shards" : 1,
          "relocating_shards" : 0,
          "initializing_shards" : 0,
          "unassigned_shards" : 0
        },
        ... (same for the other shards)

Thank you

[warkolm (Mark Walkom)]

Where is that alert coming from?

[Michele_De_Benedet]

It is in the Kibana Stack Monitoring page

I did not mention, I am using version 7.12.0

[warkolm (Mark Walkom)]

@Michele_De_Benedet

But how can it be? I am using 6 shards, each having a 10GB size. How can I fix it?

An index is made up of its component shards, and the alert only looks at index level.

I wouldn't be too worried about this to be honest, the alert isn't very smart and doesn't take into account specific use cases.