I have a setup such that I'm proxying requests over a custom domain that points to an nginx server to Elasticsearch cluster.
I have noticed that nginx is caching the data node ips as per the ttl. Now, when a node goes down or when blue green deployments happen, until the ttl is expired nginx is routing the requests to nodes that aren't active anymore.
Has anyone else faced this issue ? How do we solve this ?