Controlling the number of shard/index in Elasticsearch 5

Elastic Stack Elasticsearch
1

Hello,

Upgrading to ES 5 I've noticed that the elasticsearch.yml file has shrinked a bit.
In particular, I can't find anymore these settings among the others:

index.number_of_shards: 5
index.number_of_replicas: 1

From https://github.com/elastic/elasticsearch/issues/18073 I see that the thing is by design and one shoul call the REST API creating a index template like this:

curl -XPUT 'http://localhost:9200/_all/_settings?preserve_existing=true' -d '{
  "index.number_of_shards" : "5"
  "index.number_of_replicas" : "1"
}'

If you can confirm me that these parameters are no more controlled by the configuration file, and also what are the benefits of this different approach? Since I'm using ELK on docker containers, I can't figure out what would be the best option to set these parameters when a new Elasticsearch container is created.

Also, for my development system, I was using these values:

index.number_of_shards: 1
index.number_of_replicas: 0

Upgrading to ES5 all indexes have changed?

2

It's true. Number of shards and replicas are meant to be controlled per-index in ES5. One benefit that I've run into already is that the configuration of the server is much simpler as there are just fewer options exposed.

3

Hi loren, thanks for the reply :slight_smile:

Is it possible to change existing values?
I have a couple of indexes, and I'm trying to:

PUT /_all/_settings?preserve_existing=true
{
  "index.number_of_shards" : "1",
  "index.number_of_replicas" : "0"
}

But Elasticsearch is replying to me:

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "can't change the number of shards for an index"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "can't change the number of shards for an index"
  },
  "status": 400
}

Why?

4

You cannot change the number of shards on an existing index. You need to delete and recreate the index with the new shard count in its settings (either directly in its settings or via an index template).

Or create a new index and move an alias from the old one to the new one.

5

What about reindexing?

Consider that I'm running Elasticsearch behind Logstash, so I have daily indices creation.
The index name is logstash-*

I'm thinking of creating a new index:

PUT logstash_new-*
{
    "settings" : {
        "index" : {
            "number_of_shards" : 1, 
            "number_of_replicas" : 0 
        }
    }
}

And reindex everything to this new index:

POST _reindex
{
  "source": {
    "index": "logstash-*"
  },
  "dest": {
    "index": "logstash_new-*"
  }
}

But I don't know if it would work and if I'm getting the sintax right :neutral_face:

1 Like
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.