Hi,
I am using Logstash to feed an index. Current index size is 60GB and I'm expecting it to grow to at most 100GB, so I set it to use 6 shards. Elasticsearch is ran on a single node.
The status of the index in Kibana is as follows:
Status Health Green
Alerts 1
Total 60.9 GB
Primaries 60.9 GB
Documents 63.5m
Total shards 6
Unassigned shards 0
And the red alert says:
The following index: [index_name] has a large shard size of: 65.35GB at October 10, 2022 9:15 AM CEST
But how can it be? I am using 6 shards, each having a 10GB size. How can I fix it?
Logstash output configuration:
output {
# stdout { }
elasticsearch {
hosts => ...
user => ...
password => ...
index => "index_name"
doc_as_upsert => true
action => "update"
document_id => ...
template => "/usr/share/logstash/template/index_name.json"
template_name => "index_name"
}
}
Template configuration:
{
"index_patterns": ["index_name"],
"settings": {
"index": {
"number_of_shards": 6,
"number_of_replicas": 0
}
}
}
Get shards
index_name 4 p STARTED 10587624 10.4gb * elasticsearch
index_name 1 p STARTED 10585296 9.9gb * elasticsearch
index_name 2 p STARTED 10585089 9.9gb * elasticsearch
index_name 5 p STARTED 10584801 9.9gb * elasticsearch
index_name 3 p STARTED 10588147 10.1gb * elasticsearch
index_name 0 p STARTED 10590486 10.4gb * elasticsearch
Shard status (GET _cluster/health/?level=shards):
"index_name" : {
"status" : "green",
"number_of_shards" : 6,
"number_of_replicas" : 0,
"active_primary_shards" : 6,
"active_shards" : 6,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"shards" : {
"0" : {
"status" : "green",
"primary_active" : true,
"active_shards" : 1,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0
},
"1" : {
"status" : "green",
"primary_active" : true,
"active_shards" : 1,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0
},
... (same for the other shards)
Thank you
