As title suggests, at 4PM PST (which is 12AM on the boxes that host the cluster) my cluster goes into red state for a brief second. This happens consistently every day. I observed it last week in marvel and it seemed to be red for less than the duration of a 5 second minimum refresh. Our ES clusters in pre production environments exhibit the same behavior.
The cluster does not appear to be losing any requests during this time, but it's still of concern given that 4PM PST is a peek hour for our service. My questions are as follows:
A. is this normal behavior for an ES cluster?
B. if so, is it possible to move this reset to a different time when we know there will be less requests?