Can't updating index settings via /_settings prior to index existance

Elastic Stack Elasticsearch
1

From my understanding index.number_of_replicas can be set in the elsaticsearch.yml config yet it seems I can't update it via the /_settings API endpoint unless the index already exists:

# curl -XPUT http://127.0.01:9200/_settings -d '{"index.refresh_interval":"15s","index.number_of_replicas":1}'

{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index","resource.type":"index_expression","resource.id":"_all"}],"type":"index_not_found_exception","reason":"no such index","resource.type":"index_expression","resource.id":"_all"},"status":404}

# curl -XPOST http://127.0.0.1:9200/test/test -d '{"name": "John", "title": "Mr", "age": 36}'

{"_index":"test","_type":"test","_id":"AVY3Yy-0zDeC1CeGorHC","_version":1,"_shards":{"total":1,"successful":1,"failed":0},"created":true}

# curl -XPUT http://127.0.0.1:9200/_settings -d '{"index.refresh_interval":"15s","index.number_of_replicas":1}'

{"acknowledged":true}

The documentation for the /_settings API endpoint doesn't indicate this limitation. Considering this can be set in the elasticsearch.yml config (thus causing it to be applied before any indices exists) would it be reasonable to assume that it should be possible to apply global index settings via API prior to any index being created?

Thanks!

2

Yep, that default is not dynamically updateable. But more importantly, the ability to even specify index.number_of_replicas in the elasticsearch.yml has been removed entirely in 5.0. So the short answer is: "don't worry about it, since it's going away soon" :slight_smile:

Basically, being able to specify that default in a static config is a bad thing and we yanked it. The elasticsearch.yml configs are supposed to be for node-level configuration. Something like index.number_of_replicas is really an index-level parameter, and has ramifications based on which node becomes master, etc.

For example, what happens if the value is configured inconsistently across the nodes, and different nodes become master at different times? The default for the cluster would change.

There are more details about it here: https://github.com/elastic/elasticsearch/issues/18073#issuecomment-229579221

If you want to update all the indices at once, you can use a wildcard:

curl -XPUT http://127.0.01:9200/*/_settings {...}
or
curl -XPUT http://127.0.01:9200/_all/_settings {...}

And if you want to set a default, index templates are a much better option than the static config in elasticsearch.yml

3

Thanks for the clarification on that! I do realize that you can set these
default settings within an index template, although I was curious to know
as to why I can't just set it via the index.number_of_replicas via the
/_settings API even before any indices exist? Considering that the index
templates would technically be set prior to any index being created, I
don't see what the difference is when it comes to setting it via the
/_settings endpoint?

4

Not really a good reason to be honest. Just historical legacy / cruft. It's not technically a cluster-level setting based on how it works, so you can't set it dynamically via /_cluster/settings. It's technically an index-level setting... but you can only update index-level settings through the API on an actual, existing index. Which is why you cannot set it before the index is created.

So it occupies this bizzaro role of being an index-level-but-also-cluster-level setting, which happens to only be configurable via the static config.

It really was just a hacky thing put in place a loooong time ago, which didn't follow the normal semantics of how settings should work. Which is why it's a bit of a special snowflake, and also why it's been removed :slight_smile:

5

Ok, this makes more sense now. Thanks again for the clarification.

6

Sure, no problem! It's definitely an odd duck, you weren't wrong in thinking its behavior strange :slight_smile: