Elasticsearch 8 single node replicas 0

Elastic Stack Elasticsearch
1

I have a single node cluster which is showing yellow because replicas are set to 1

Is there a way to get this setup so that all future indexes have replicas set to 0

This used to be easy with legacy templates

curl -XPUT localhost:9200/_template/default -d '{"index_patterns": ["*"], "order": "1", "settings": {"number_of_shards": "1","number_of_replicas": "0"}}' -H 'Content-Type: application/json'

Appreciate the help

image
image1472×759 91.5 KB

1 Like
2

Is that okay?

PUT /_index_template/replicas
{
  "index_patterns" : ["*"],
  "template": {
    "settings" : {
      "number_of_replicas" : 0
    }
  }
}
1 Like
3 
curl -XPUT -k -u elastic:$pass https://localhost:9200/_index_template/replicas -d '{"index_patterns": ["*"], "template": {"settings": {"number_of_replicas": "0"}}}' -H 'Content-Type: application/json'

Gives the following error, maybe I need to change priority to something higher

{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"index template [replicas] has index patterns [*] matching patterns from existing templates [.kibana-event-log-8.1.0-template,.kibana_security_session_index_template_1,.monitoring-beats-mb,.monitoring-ent-search-mb,.monitoring-es-mb,.monitoring-kibana-mb,.monitoring-logstash-mb] with patterns (.kibana-event-log-8.1.0-template => [.kibana-event-log-8.1.0-*],.kibana_security_session_index_template_1 => [.kibana_security_session_1],.monitoring-beats-mb => [.monitoring-beats-8-*],.monitoring-ent-search-mb => [.monitoring-ent-search-8-*],.monitoring-es-mb => [.monitoring-es-8-*],.monitoring-kibana-mb => [.monitoring-kibana-8-*],.monitoring-logstash-mb => [.monitoring-logstash-8-*]) that have the same priority [0], multiple index templates may not match during index creation, please use a different priority"}],"type":"illegal_argument_exception","reason":"index template [replicas] has index patterns [*] matching patterns from existing templates [.kibana-event-log-8.1.0-template,.kibana_security_session_index_template_1,.monitoring-beats-mb,.monitoring-ent-search-mb,.monitoring-es-mb,.monitoring-kibana-mb,.monitoring-logstash-mb] with patterns (.kibana-event-log-8.1.0-template => [.kibana-event-log-8.1.0-*],.kibana_security_session_index_template_1 => [.kibana_security_session_1],.monitoring-beats-mb => [.monitoring-beats-8-*],.monitoring-ent-search-mb => [.monitoring-ent-search-8-*],.monitoring-es-mb => [.monitoring-es-8-*],.monitoring-kibana-mb => [.monitoring-kibana-8-*],.monitoring-logstash-mb => [.monitoring-logstash-8-*]) that have the same priority [0], multiple index templates may not match during index creation, please use a different priority"},"status":400}

I can change the existing settings with this command, but when 8.2.0 comes out I have a feeling the new indexes and or streams will have the default replica of 1

curl -XPUT -k -u elastic:$pass https://localhost:9200/*/_settings?pretty -H 'Content-Type: application/json' -d '{"settings": {"number_of_replicas": 0}}'
{
  "acknowledged" : true
}
4

As you said, you can use priority to set template priority

5

Still need to find out what this means, I'd like to keep things "default" apart from the replica number

curl -XPUT -k -u elastic:$pass https://localhost:9200/_index_template/replicas -d '{"index_patterns": ["*"], "template": {"settings": {"number_of_replicas": "0"}},"priority": 7}' -H 'Content-Type: application/json'

{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"composable template [replicas] with index patterns [], priority [7] and no data stream configuration would cause data streams [.monitoring-es-8-mb] to no longer match a data stream template"}],"type":"illegal_argument_exception","reason":"composable template [replicas] with index patterns [], priority [7] and no data stream configuration would cause data streams [.monitoring-es-8-mb] to no longer match a data stream template"},"status":400}

6

Hi all,

I have the same problem, I have a single node with 0 replica and since ELK 8.0 all of my indexes are in yellow state. I follow this fix to resolve the problem and he work except for data streams. Setting index.number_of_replicas => 0 by default in ES >= 5. I manually index data streams to recover a green state. Any help would be appreciate to auto indexing the data streams.

Thanks.

7

Can I ask you to elaborate on manually indexing data streams?
Replicas are just copies of shards so if they are discarded it doesn't matter (in a single node cluster)
This command goes through the existing indices and changes them to green

curl -XPUT -k -u elastic:$pass https://localhost:9200/*/_settings?pretty -H 'Content-Type: application/json' -d '{"settings": {"number_of_replicas": 0}}'

Today I created a template with a high priority that stuck

curl -XGET -k -u elastic:$pass https://localhost:9200/_index_template/default?pretty
{
  "index_templates" : [
    {
      "name" : "default",
      "index_template" : {
        "index_patterns" : [
          "*"
        ],
        "template" : {
          "settings" : {
            "index" : {
              "number_of_replicas" : "0"
            }
          }
        },
        "composed_of" : [ ],
        "priority" : 90000000,
        "_meta" : {
          "description" : "this used to be easy"
        },
        "data_stream" : {
          "hidden" : false,
          "allow_custom_routing" : false
        }
      }
    }
  ]
}

It seems I can't delete it anymore as it's in use now heh

curl -XDELETE -k -u elastic:$pass https://localhost:9200/_index_template/default?pretty
{
  "error" : {
    "root_cause" : [
      {
        "type" : "illegal_argument_exception",
        "reason" : "unable to remove composable templates [default] as they are in use by a data streams [.monitoring-es-8-mb, logs-elastic_agent-default, logs-elastic_agent.filebeat-default, logs-elastic_agent.fleet_server-default, logs-elastic_agent.metricbeat-default, logs-system.application-default, logs-system.auth-default, logs-system.security-default, logs-system.syslog-default, logs-system.system-default, metricbeat-8.1.0, metricbeat-8.1.1, metrics-elastic_agent.elastic_agent-default, metrics-elastic_agent.filebeat-default, metrics-elastic_agent.fleet_server-default, metrics-elastic_agent.metricbeat-default, metrics-system.cpu-default, metrics-system.diskio-default, metrics-system.filesystem-default, metrics-system.fsstat-default, metrics-system.load-default, metrics-system.memory-default, metrics-system.network-default, metrics-system.process-default, metrics-system.process.summary-default, metrics-system.socket_summary-default, metrics-system.uptime-default]"
      }
    ],
    "type" : "illegal_argument_exception",
    "reason" : "unable to remove composable templates [default] as they are in use by a data streams [.monitoring-es-8-mb, logs-elastic_agent-default, logs-elastic_agent.filebeat-default, logs-elastic_agent.fleet_server-default, logs-elastic_agent.metricbeat-default, logs-system.application-default, logs-system.auth-default, logs-system.security-default, logs-system.syslog-default, logs-system.system-default, metricbeat-8.1.0, metricbeat-8.1.1, metrics-elastic_agent.elastic_agent-default, metrics-elastic_agent.filebeat-default, metrics-elastic_agent.fleet_server-default, metrics-elastic_agent.metricbeat-default, metrics-system.cpu-default, metrics-system.diskio-default, metrics-system.filesystem-default, metrics-system.fsstat-default, metrics-system.load-default, metrics-system.memory-default, metrics-system.network-default, metrics-system.process-default, metrics-system.process.summary-default, metrics-system.socket_summary-default, metrics-system.uptime-default]"
  },
  "status" : 400
}

It looks promising but I need to find out how to actually delete it
that's for another day...
(Index Template cannot be deleted / error message seems incorrect · Issue #84171 · elastic/elasticsearch · GitHub)

8

I just tried another thing. In the Kibana Web interface I edit the template of index who not auto indexing. I just add a line with the setting "number_of_replicas": "0". I tell you in few days if this work.

monitoring-es-mb

9

Nice work, I've been playing with those too- in Component Templates I do a search for settings and changed them all

With my setup I've had issues with filebeat creating a new index with every upgrade with replica of 1 (filebeat-8.0.0, filebeat-8.0.1,etc)

I've realized last night these come from the filebeat.yml so I specifically added replicas zero there
I can't really test unless I delete the data :confused: or wait until next release :slight_smile:

# ======================= Elasticsearch template setting =======================
setup.template.settings:
  index.number_of_shards: 1
  index.number_of_replicas: 0

Setting up a default with the following settings didn't work for filebeat possible because I haven't made the above change, even though default template is getting picked up

curl -XPUT -k -u elastic:$pass https://localhost:9200/_index_template/default -d '{"index_patterns":["*"],"template":{"settings":{"index":{"number_of_replicas":"0"}}},"composed_of":[],"priority":90000000,"_meta":{"description":"b7-default"},"data_stream":{"hidden":false,"allow_custom_routing":false}}}' -H 'Content-Type: application/json'

image

I tried to delete the template I made by changing the default template from * to something not existent as was suggested in the previous post (github link) but I was unable to because .fleet-actions-results doesn't have a template, so I had to create it with a priority of 90000001

In any case I don't recommend doing any of this, everything I read so far suggests that setting up a template with * is a bad thing, and looks like setting it up with * and negating the . (to not include hidden indices in the template is not working, there's a github issue open for that.
It would be so much simpler if there was something in Elasticsearch.yml to set replica settings globally :smiley: and not have to worry about it

curl -XPUT -k -u elastic:$pass https://localhost:9200/_index_template/default -d '{"index_patterns":["whatever"],"template":{"settings":{"index":{"number_of_replicas":"0"}}},"composed_of":[],"priority":90000000,"_meta":{"description":"b7"},"data_stream":{"hidden":false,"allow_custom_routing":false}}}' -H 'Content-Type: application/json'
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"composable template [default] with index patterns [b7], priority [90000000] would cause data streams [.fleet-actions-results] to no longer match a data stream template"}],"type":"illegal_argument_exception","reason":"composable template [default] with index patterns [b7], priority [90000000] would cause data streams [.fleet-actions-results] to no longer match a data stream template"},"status":400}
10

Hi,

I have check this morning in the Web interface of Kibana and all of my index are in green state, data streams too so it seems modify the template in the Web interface fix the issue.

1 Like
11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.