Elasticsearch service restart - rebuilds kibana indexes?

I have a Chef recipe that calls to restart elasticsearch service whenever certain things change. Primarily if the yaml config changes. I've noticed that sometimes this causes my cluster health to turn yellow or red. The alert says that the Kibana indexes are rebuilding. After 5-10 minutes everything goes back to normal.

I feel like I'm doing something wrong. Are there normal circumstances when the Kibana index would need to be rebuilt like this? Maybe a softer way to restart elasticsearch to prevent this?

Check out https://www.elastic.co/guide/en/elasticsearch/guide/2.x/_rolling_restarts.html