Setting index.number_of_replicas => 0 by default in ES >= 5

Elastic Stack Elasticsearch
1

I am trying to build a single-node all-in-one ELK 6 cluster for development and testing. To do this and get a cluster that goes into green state, I need to ensure that all indices inherit a setting index.number_of_replicas => 0.

I have read threads on this site that explained that after ES >= 5, it is no longer possible to define this setting in the elasticsearch.yml. And I know how to manually change the index settings using the REST API.

But how can I do this in an automated way now? I don't know the names of the indices until the ELK cluster builds.

After building I see a bunch of templates:

$ curl 0.0.0.0:9200/_template | jq keys
[
  ".ml-anomalies-",
  ".ml-meta",
  ".ml-notifications",
  ".ml-state",
  ".monitoring-alerts",
  ".monitoring-beats",
  ".monitoring-es",
  ".monitoring-kibana",
  ".monitoring-logstash",
  ".triggered_watches",
  ".watch-history-7",
  ".watches",
  "logstash",
  "logstash-index-template",
  "security-index-template",
  "security_audit_log"
]

and one index for logstash:

$ curl 0.0.0.0:9200/_cat/indices?v
health status index               uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   logstash-2018.07.02 ppxqmtMCR5SZUmXMwdVJtA   5   1       3114            0      1.1mb          1.1mb

Many thanks.

2

To prevent new indices from having replica shards you will need to create an Index template in your 1-node cluster, which is automatically applied during index creation.

For instance, if you want to set the number of replicas to 0 for new logstash indices you will have to add something like this:

curl -XPUT "localhost:9200/_template/logstash_template" -H 'Content-Type: application/json' -d'
{
  "index_patterns": ["logstash*"],
  "settings": {
    "number_of_replicas": 0
  }
}
'

The crucial element here is "index_patterns" which must match the start of the index names. If you want this setting to work for all indices (though I haven't tried this myself), you should be able to do:

curl -XPUT "localhost:9200/_template/default_template" -H 'Content-Type: application/json' -d'
{
  "index_patterns": ["*"],
  "settings": {
    "number_of_replicas": 0
  }
}
'

Here the asterisk should match all new index names.

3

Right, that did fix it, although for the record the command was:

curl -XPUT "localhost:9200/_template/default_template" -H 'Content-Type: application/json' -d'
{
  "index_patterns": ["*"],
  "settings": {
    "index": {
      "number_of_replicas": 0
    }
  }
}
'

Key point (for me anyway) is that this needs to be issued before Logstash starts, as it won't affect any templates already created. (I had tried it earlier with the Puppet module and an ordering problem meant that it ran after Logstash started.)