Details/Insights on how switching the index behind an alias works

I would like to get some insight information about how an alias switch works.

Our scenario

We observed a scenario in our cluster (only 3 nodes) where we did an alias switch and still queried the old data from the old index for a couple of minutes before we got the new one. There was no load on the cluster during that time. Our indices have about 200k documents and are roughly 200 MB in size. Each index has one primary shard and two replicas.
We switched the index behind the alias to the newly build index and found out that we only got the new data after about 1-2 minutes.

Our theory is that the alias was not immediate and took some time (for whatever reason).

Can somebody explain in a bit more detail what happens when the alias is switched to a new index?

More or less, a new index is created in the cluster state. The alias is changed in the cluster state.

The cluster state is broadcasted to the nodes.

What is the output of:

GET /
GET /_cat/nodes?v
GET /_cat/health?v
GET /_cat/indices?v

If some outputs are too big, please share them on gist.github.com and link them here.

1 Like

Hey David,

here is the requested output:

I would like to add that this scenario happens very rarely in our cluster. And sadly we cannot replicate it. So this is very hard to debug.

I don't see anything wrong here.
I have no idea why you are seeing such a thing. :pensive:

Thanks for taking a look at this David!

We will monitor our cluster closely and see if this happens again. Maybe we can share some more information when it happens again.