Creating index template anytime regardless cluster health status?

[lukas_vlcek]

Hi,

AFAIK index templates are stored in cluster state, are there any conditions that need to be met and should be checked before client creates a new index template? Especially right after the cluster is started does it make sense to wait for the yellow cluster health status or it is ok (and recommended) to push index templates into ES as soon as it can accept relevant REST endpoint requests? If there are any conditions that should be met before creating index templates are they the same for ES 1.x, 2.x and possible 5.x?

Regards,
Lukáš

[warkolm (Mark Walkom)]

You can send templates as soon as ES is listening, though it makes sense to wait for it to be green.

[lukas_vlcek]

Hi Mark,

"you can" means this is the official recommendation?

Specifically, I am talking about the situation where node is being pushed index template as part of node bring up script. Is it still recommended to not wait for any specific cluster health status or at least expected number of nodes in the cluster before index template is pushed?

BTW, waiting for green sounds tricky to me, especially in cases where there is index with higher replication factor than number of expected nodes+1. I thought that yellow should be better? But still the cluster health status is solely relevant to state of index shards and replicas but index templates are not stored in cluster indices, it is stored in cluster state and its consistency is not reflected in cluster health status, or is it?

I am not aware of how the cluster state is synced between individual nodes but there isn't any public API that I can use to check the status of this sync operation, is there? If node joins the cluster I assume it won't accept public API calls until it "thinks" the cluster state has been synced, right? If this is the case than may be it would make sense to wait until the cluster has expected number of nodes before index templates are pushed in, WDYT?

Still looking for any official documentation on cluster state consistency. The only note I have found is https://www.elastic.co/guide/en/elasticsearch/reference/2.3/modules-discovery-zen.html#_cluster_state_updates

Regards,
Lukáš

[warkolm (Mark Walkom)]

Waiting till green seems sane. However....

@lukas_vlcek

BTW, waiting for green sounds tricky to me, especially in cases where there is index with higher replication factor than number of expected nodes+1.

Yes, yellow in that case seems the best in this situation.

@lukas_vlcek

I am not aware of how the cluster state is synced between individual nodes but there isn't any public API that I can use to check the status of this sync operation, is there?

Nope.

@lukas_vlcek

If node joins the cluster I assume it won't accept public API calls until it "thinks" the cluster state has been synced, right?

On a high level, yes.

@lukas_vlcek

If this is the case than may be it would make sense to wait until the cluster has expected number of nodes before index templates are pushed in, WDYT?

If you have expected nodes set, then yes, definitely.

We don't really recommend anything for this specific situation because, "It Depends", as you can see
But a non-red status would be the lowest requirement I'd think.