Problem with Number of Shards (8 to 5)

Elastic Stack Elasticsearch
1

I upgraded my elasticsearch 1.3.2 to elasticsearch 2.3.2. All was well, but due to certain reasons I had to downgrade it back to 1.3.2. Now, I get to see only 5 shards per index. It used to be 8 shards per index.

I have no idea how this has happened. Can anyone tell me how to go back to having 8 shards per index?

2

Data isn't really backwards compatible. But If you hadn't written anything it might be ok.

That is a thing that you can get either from elasticsearch.yaml or by setting up a template.

3

I did try updating the elasticsearch.yml file with 8 (for the number of shards) and tried restarting everything. But it doesn't give me back my 8 shards. It just shows 5.

4

That setting is a default for newly created indexes. When you create a new index does it come with the 8?

5

We just have one index in this cluster. The number of shards has always been 8. (Now, I wonder how they set it up so it could produce 8 shards. I see this set nowhere. Not even in the elasticsearch.yml file. Not even in bin/setenv.) After the upgrade-downgrade exercise I've done, the number of shards has dipped to 5. I've tried various things, like setting it up in the .yml file, using a template as you suggested. Nothing works!

I just need some direction as to how to go back to having 8 shards.

6

I have used a template like this. Can you tell me if it's the right way?

curl -XPUT local:9200/_template/template_1 -d ' {
"template" : "me*"
"settings": {
"number_of_shards": 8
}
}'

Should I add anything else?

7

Something like that will work, sure. You'll have to recreate the index if you want to change the number of shards it has. That is a thing that is set at creation. You can use the reindex command to copy everything from the old index into the new index if you'd like.

8

Can you tell me how to reindex? I've read the documentation here: https://www.elastic.co/guide/en/elasticsearch/guide/1.x/reindex.html

What I haven't been able to understand is this: "To reindex all of the documents from the old index efficiently, use scan-and-scroll to retrieve batches of documents from the old index, and the bulk API to push them into the new index."

I did get the scroll ID. But how do I use it to insert documents into my new index using a curl command(in the bulk API to be specific)? I haven't found any examples of this. Can you help?