# How to set shards based on size instead of date?

**URL:** <https://discuss.elastic.co/t/how-to-set-shards-based-on-size-instead-of-date/146250>\
**Category:** Elasticsearch\
**Created:** [August 27, 2018, 11:14pm UTC](https://discuss.elastic.co/t/how-to-set-shards-based-on-size-instead-of-date/146250 "2018-08-27T23:14:55Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![robhep](https://avatars.discourse-cdn.com/v4/letter/r/85e7bf/32.png) [@robhep](https://discuss.elastic.co/u/robhep)\
**Post date:** [August 27, 2018, 11:14pm UTC](https://discuss.elastic.co/t/how-to-set-shards-based-on-size-instead-of-date/146250/1 "2018-08-27T23:14:55Z")

</div>

Hi, I have a 5 node cluster in a lab environment. 2 data nodes, 3 master eligible nodes, 3 kibana, and 2 logstash. It's a low volume setup with about a half a dozen systems feed data to it. It has the default setup with 5 primary and 1 replicate shards per index. However my indices are about 2-8 mb in size, so for about 36 days, I have a ton of shards. How can I reduce the number of primary shards down to 1 primary and 1 replicate?

{  
"cluster\_name" : "ELK-cluster",  
"status" : "green",  
"timed\_out" : false,  
"number\_of\_nodes" : 5,  
"number\_of\_data\_nodes" : 2,  
"active\_primary\_shards" : 415,  
"active\_shards" : 830,  
"relocating\_shards" : 0,  
"initializing\_shards" : 0,  
"unassigned\_shards" : 0,  
"delayed\_unassigned\_shards" : 0,  
"number\_of\_pending\_tasks" : 0,  
"number\_of\_in\_flight\_fetch" : 0,  
"task\_max\_waiting\_in\_queue\_millis" : 0,  
"active\_shards\_percent\_as\_number" : 100.0  
}

Thanks,  
Robert

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 27, 2018, 11:34pm UTC](https://discuss.elastic.co/t/how-to-set-shards-based-on-size-instead-of-date/146250/2 "2018-08-27T23:34:11Z")

</div>

You can update the default Logstash template so that it only uses a single shard, that's probably the easiest way.

---

<div class="post-metadata">

**Author:** ![robhep](https://avatars.discourse-cdn.com/v4/letter/r/85e7bf/32.png) [@robhep](https://discuss.elastic.co/u/robhep)\
**Post date:** [August 28, 2018, 3:25am UTC](https://discuss.elastic.co/t/how-to-set-shards-based-on-size-instead-of-date/146250/3 "2018-08-28T03:25:00Z")

</div>

Thanks for the suggestion.  
I have a Ossec/Wazuh manager server with filebeat sending alerts to Logstash. I'm not sure where to add the line for defining the number of shards, primary and replica. This is the template from the Wazuh group.

cat /etc/logstash/conf.d/01-wazuh.conf  
Wazuh - Logstash configuration file  
Remote Wazuh Manager - Filebeat input  
input {  
beats {  
port =\> 5000  
codec =\> "json\_lines"  
ssl =\> true  
ssl\_certificate =\> "/etc/logstash/logstash.crt"  
ssl\_key =\> "/etc/logstash/logstash.key"  
}  
}  
filter {  
if [data][srcip] {  
mutate {  
add\_field =\> ["@src\_ip", "%{[data][srcip]}" ]  
}  
}  
if [data][aws][sourceIPAddress] {  
mutate {  
add\_field =\> ["@src\_ip", "%{[data][aws][sourceIPAddress]}" ]  
}  
}  
}  
filter {  
geoip {  
source =\> "@src\_ip"  
target =\> "GeoLocation"  
fields =\> ["city\_name", "country\_name", "region\_name", "location"]  
}  
date {  
match =\> ["timestamp", "ISO8601"]  
target =\> "@timestamp"  
}  
mutate {  
remove\_field =\> ["timestamp", "beat", "input\_type", "tags", "count", "@version", "log", "offset", "type", "@src\_ip", "host"]  
}  
}  
output {  
elasticsearch {  
hosts =\> ["localhost:9200"]  
index =\> "wazuh-alerts-3.x-%{+YYYY.MM.dd}"  
"number\_of\_shards" : 1 \<-- here?  
"number\_of\_replicas" : 1 \<-- here?  
document\_type =\> "wazuh"  
}  
}

Link for above from Wazuh Documentation.  
curl -so /etc/logstash/conf.d/01-wazuh.conf [https://raw.githubusercontent.com/wazuh/wazuh/3.5/extensions/logstash/01-wazuh-remote.conf](https://raw.githubusercontent.com/wazuh/wazuh/3.5/extensions/logstash/01-wazuh-remote.conf)

Thanks,  
Robert

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 28, 2018, 5:38am UTC](https://discuss.elastic.co/t/how-to-set-shards-based-on-size-instead-of-date/146250/4 "2018-08-28T05:38:13Z")

</div>

If you want to change the number of primary shards and replicas, you need to do so through an [https://www.elastic.co/guide/en/elasticsearch/reference/6.4/indices-templates.html](https://www.elastic.co/guide/en/elasticsearch/reference/6.4/indices-templates.html). To get even more flexibility and cut indices based on size and/or age, you could switch to using the [rollover index API](https://www.elastic.co/guide/en/elasticsearch/reference/6.4/indices-rollover-index.html), which is also described in [this blog post](https://www.elastic.co/blog/managing-time-based-indices-efficiently).

---

<div class="post-metadata">

**Author:** ![robhep](https://avatars.discourse-cdn.com/v4/letter/r/85e7bf/32.png) [@robhep](https://discuss.elastic.co/u/robhep)\
**Post date:** [August 28, 2018, 8:39pm UTC](https://discuss.elastic.co/t/how-to-set-shards-based-on-size-instead-of-date/146250/5 "2018-08-28T20:39:45Z")

</div>

Thanks Christian.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 25, 2018, 8:39pm UTC](https://discuss.elastic.co/t/how-to-set-shards-based-on-size-instead-of-date/146250/6 "2018-09-25T20:39:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.