# Setting the number of shards and replicas in /etc/filebeat/filebeat.yml

**URL:** https://discuss.elastic.co/t/setting-the-number-of-shards-and-replicas-in-etc-filebeat-filebeat-yml/302029
**Category:** Beats
**Tags:** filebeat
**Created:** [April 9, 2022, 1:35pm UTC](https://discuss.elastic.co/t/setting-the-number-of-shards-and-replicas-in-etc-filebeat-filebeat-yml/302029 "2022-04-09T13:35:12Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![ristov1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ristov1/32/104197_2.png) [@ristov1](https://discuss.elastic.co/u/ristov1)
#### Post date: [April 9, 2022, 1:35pm UTC](https://discuss.elastic.co/t/setting-the-number-of-shards-and-replicas-in-etc-filebeat-filebeat-yml/302029/1 "2022-04-09T13:35:12Z")

</div>

According to filebeat documentation ([Configure Elasticsearch index template loading | Filebeat Reference [8.1] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-template.html)), it should be possible to set the number of shards and replicas for Filebeat indexes with the following settings:

```auto
setup.template.settings:
  index.number_of_shards: 1
  index.number_of_replicas: 0

```

I have tried these settings together with

```auto
setup.template.overwrite: true

```

which should set the number of shards to 1 and the number of replicas to 0.

However, when running filebeat-7.16.3 as follows

```auto
filebeat -e setup --pipelines --index-management --modules microsoft

```

in order to set up pipelines and index management in Elasticsearch, the above settings have no effect whatsoever (even after removing the filebeat index templates, pipelines and ilm policy by hand).

Is it a know bug of filebeat?

--  
risto

---

<div class="post-metadata">

### Author: ![zx8086](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zx8086/32/94917_2.png) [@zx8086](https://discuss.elastic.co/u/zx8086)
#### Post date: [April 9, 2022, 2:30pm UTC](https://discuss.elastic.co/t/setting-the-number-of-shards-and-replicas-in-etc-filebeat-filebeat-yml/302029/2 "2022-04-09T14:30:34Z")

</div>

What about setting the replica number in template instead?

```auto
PUT /<Index / template>/_settings
{
  "index" : {
    "number_of_replicas" : 0
  }
}

```

---

<div class="post-metadata">

### Author: ![ristov1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ristov1/32/104197_2.png) [@ristov1](https://discuss.elastic.co/u/ristov1)
#### Post date: [April 9, 2022, 7:10pm UTC](https://discuss.elastic.co/t/setting-the-number-of-shards-and-replicas-in-etc-filebeat-filebeat-yml/302029/3 "2022-04-09T19:10:34Z")

</div>

I know that it can be done, but I am rather interested why setting the number of shards and replicas in the filebeat configuration file does not work, although the documentation advertises it.  
risto

---

<div class="post-metadata">

### Author: ![zx8086](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zx8086/32/94917_2.png) [@zx8086](https://discuss.elastic.co/u/zx8086)
#### Post date: [April 9, 2022, 7:31pm UTC](https://discuss.elastic.co/t/setting-the-number-of-shards-and-replicas-in-etc-filebeat-filebeat-yml/302029/4 "2022-04-09T19:31:28Z")

</div>

For that answer you need to share more of the filebeat.yml ...

---

<div class="post-metadata">

### Author: ![ristov1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ristov1/32/104197_2.png) [@ristov1](https://discuss.elastic.co/u/ristov1)
#### Post date: [April 9, 2022, 8:31pm UTC](https://discuss.elastic.co/t/setting-the-number-of-shards-and-replicas-in-etc-filebeat-filebeat-yml/302029/5 "2022-04-09T20:31:15Z")

</div>

Here is the relevant section from the configuration file:

```auto
setup.ilm.overwrite: true

setup.template.overwrite: true

setup.template.settings:
  index.number_of_shards: 2
  index.number_of_replicas: 0

```

The other parts in the configuration file set up access to Elasticsearch and it works without issues -- after deleting the template, it appears again when 'filebeat setup' gets executed. However, what is _not_ working as expected are the directives for configuring the number of shards and replicas (they will both remain set to 1 which is the default).

---

<div class="post-metadata">

### Author: ![zx8086](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zx8086/32/94917_2.png) [@zx8086](https://discuss.elastic.co/u/zx8086)
#### Post date: [April 9, 2022, 8:44pm UTC](https://discuss.elastic.co/t/setting-the-number-of-shards-and-replicas-in-etc-filebeat-filebeat-yml/302029/6 "2022-04-09T20:44:33Z")

</div>

After running the setup command ?

```auto
sudo filebeat setup -e --strict.perms=false \
  -E output.logstash.enabled=false \
  -E output.elasticsearch.hosts=['https://0.0.0.0:9200'] \
  -E output.elasticsearch.username=elastic \
  -E output.elasticsearch.password=changeme \
  -E setup.kibana.host=https://0.0.0.0:5601

```

---

<div class="post-metadata">

### Author: ![ristov1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ristov1/32/104197_2.png) [@ristov1](https://discuss.elastic.co/u/ristov1)
#### Post date: [April 10, 2022, 9:49am UTC](https://discuss.elastic.co/t/setting-the-number-of-shards-and-replicas-in-etc-filebeat-filebeat-yml/302029/7 "2022-04-10T09:49:20Z")

</div>

I think I have found the reason for this issue. I needed to change the number of replicas from 1 to 0, and in addition to my setup.template.settings entry, the filebeat.yml file contained another entry that looked like this:

```auto
setup.template.settings:
  index.number_of_shards: 1

```

As you can see, It specifies just the number of shards without the number of replicas. Initially, I didn't notice the second instance and even though the number of replicas was not provided there, it appears that the mere appearance of the second setup.template.settings: directive will override the previous number\_of\_replicas setting with the default (that is, 1). After getting rid of the duplicate block my original command line worked as expected.

risto

---

<div class="post-metadata">

### Author: ![zx8086](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zx8086/32/94917_2.png) [@zx8086](https://discuss.elastic.co/u/zx8086)
#### Post date: [April 10, 2022, 9:53am UTC](https://discuss.elastic.co/t/setting-the-number-of-shards-and-replicas-in-etc-filebeat-filebeat-yml/302029/8 "2022-04-10T09:53:20Z")

</div>

Usually that is why its better to share your full configuration for eagle eyes to spot these thing.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 8, 2022, 11:53am UTC](https://discuss.elastic.co/t/setting-the-number-of-shards-and-replicas-in-etc-filebeat-filebeat-yml/302029/9 "2022-05-08T11:53:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
