Way to make a shard completely read-only?

Elastic Stack Elasticsearch
1

Hi,

I'm running a ~30TB ES cluster on v1.7.5 and we try to make recoveries faster. We have several indices, but only one receives writes at a time. Other are "cold", they only receive reads. They've already been optimized to 1 segment.

We noticed that as expected, recoveries are near instant when the shard has a "sync_id" key and it's identical to the primary shards they're replicated from. So we'd like shards on "cold" indices to be read-only so when they get a "sync_id" and keep it.

In practice very old indices are not requested that much and they keep the same sync_id forever. On more recent indices I see the "sync_id" disappear and the "generation" ID is incremented (as seen with /_stats/commit?level=shards). I guess it's because Lucene (or ES?) does some internal optimizations depending on queries load.

Is there a way to freeze shards with some of the index settings? I tried "index.blocks.metadata=true" but unfortunately it breaks some "cat" APIs (at least /_cat/indices). I also tried "index.blocks.read_only=true" but it's not enough.

Thanks,

Jean-Baptiste

2

You need to upgrade to 2.X to get synced flush. Otherwise you cannot set a shard read only, only an index.

3

Our client code is not ready for 2.0 unfortunately. We already issue regular "/_flush/synced", which works well on 1.7.5. And setting the index as read-only is not enough as stated above.

Do you mean that in 2.0 issuing a "/_flush/synced" would improve the situation regarding indices being "really" read-only?

4

Different API - https://www.elastic.co/guide/en/elasticsearch/reference/2.4/indices-synced-flush.html

5

This doc doesn't differ from what was already here in 1.7.x: https://www.elastic.co/guide/en/elasticsearch/reference/1.7/indices-synced-flush.html ; but sure, I imagine internals could have changed, bugs may have been fixed, so I'll upgrade as soon as possible.

Other ideas welcome!