# Does elasticsearch automatically close replicated indices? **URL:** \ **Category:** Elasticsearch\ **Tags:** ccr-cross-cluster-replication\ **Created:** [October 24, 2019, 8:49am UTC](https://discuss.elastic.co/t/does-elasticsearch-automatically-close-replicated-indices/205019 "2019-10-24T08:49:39Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![kley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kley/32/43533_2.png) [@kley](https://discuss.elastic.co/u/kley)\ **Post date:** [October 24, 2019, 8:49am UTC](https://discuss.elastic.co/t/does-elasticsearch-automatically-close-replicated-indices/205019/1 "2019-10-24T08:49:39Z")
Hey everybody 👋 I have a new question about the behavior of elasticsearch when it comes to cross cluster replication. First up, I cannot reproduce the issue, and I don't see any logs regarding an API call for closing an index in the elastic logs. So, for whatever reason, all our follower indices got closed after we restarted elasticsearch (it seems that it is connected to the restart) - happened on both datacenters (bi-directional setup): ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1aebfdf8dcf19b7765111013e8eb1c201610f37b.png) My question is, is there a situation, where elasticsearch actively closes indices? As I said, I did not find any loglines which state that they got closed by an API call or whatsoever. Please halp ☹ ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [October 24, 2019, 9:03am UTC](https://discuss.elastic.co/t/does-elasticsearch-automatically-close-replicated-indices/205019/2 "2019-10-24T09:03:42Z")
Which version are you using? Can you share all the logs going back before this occurred (i.e. before and after the restart)? I can set up a private upload area if you'd rather not share them in public. ---
**Author:** ![kley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kley/32/43533_2.png) [@kley](https://discuss.elastic.co/u/kley)\ **Post date:** [October 24, 2019, 9:04am UTC](https://discuss.elastic.co/t/does-elasticsearch-automatically-close-replicated-indices/205019/3 "2019-10-24T09:04:48Z")
Oh yea, we are using 6.8.x. Yes please, private would be nice! ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [October 24, 2019, 9:10am UTC](https://discuss.elastic.co/t/does-elasticsearch-automatically-close-replicated-indices/205019/4 "2019-10-24T09:10:03Z")
Sure, no probs, you can upload things here: [`https://upload-staging.elastic.dev/u/619d5b64-bbae-483c-9a86-883e9c767804`](https://upload-staging.elastic.dev/u/619d5b64-bbae-483c-9a86-883e9c767804) ---
**Author:** ![kley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kley/32/43533_2.png) [@kley](https://discuss.elastic.co/u/kley)\ **Post date:** [October 24, 2019, 9:31am UTC](https://discuss.elastic.co/t/does-elasticsearch-automatically-close-replicated-indices/205019/5 "2019-10-24T09:31:18Z")
upload done. Its the logs of each dc and each node ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [October 24, 2019, 10:17am UTC](https://discuss.elastic.co/t/does-elasticsearch-automatically-close-replicated-indices/205019/6 "2019-10-24T10:17:05Z")
Thanks. These logs contain entries from `2019-10-17T07:57:38,540` until `2019-10-17T19:57:51,579`, so a few hours about a week ago Is this when the problematic restart happened? ---
**Author:** ![kley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kley/32/43533_2.png) [@kley](https://discuss.elastic.co/u/kley)\ **Post date:** [October 24, 2019, 10:51am UTC](https://discuss.elastic.co/t/does-elasticsearch-automatically-close-replicated-indices/205019/7 "2019-10-24T10:51:22Z")
exactly, that was the day all started The first ClusterBlockException we encountered on client side was around 09:23 UTC that day: ![image](https://us1.discourse-cdn.com/elastic/original/3X/3/8/3856608553170abb991a7b29caf2f89396e0bb69.png) ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [October 24, 2019, 10:57am UTC](https://discuss.elastic.co/t/does-elasticsearch-automatically-close-replicated-indices/205019/8 "2019-10-24T10:57:56Z")
Are the timestamps in the logs in UTC? The first indication that a node was restarted was logged with a timestamp of `2019-10-17T10:59:05,317`. ---
**Author:** ![kley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kley/32/43533_2.png) [@kley](https://discuss.elastic.co/u/kley)\ **Post date:** [October 24, 2019, 11:02am UTC](https://discuss.elastic.co/t/does-elasticsearch-automatically-close-replicated-indices/205019/9 "2019-10-24T11:02:45Z")
Yes they are utc. Hmm interesting, I missed that. So it seems that at least one index was closed before the restart already - and there is only one log-entry which states that... I can tell for sure, that the ClusterBlockException occured way more after the restart. Nevertheless, shouldn't there be a log entry if an index gets closed? ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [October 24, 2019, 11:19am UTC](https://discuss.elastic.co/t/does-elasticsearch-automatically-close-replicated-indices/205019/10 "2019-10-24T11:19:02Z")
Yes, looking at the code I cannot see a path to closing an index that doesn't involve logging a message containing the string `closing indices`. Your second screenshot mentions `ClusterBlockException[blocked by : [FORBIDDEN/4/index closed];]` at `06:19`, which precedes the first entry in the log files you've shared (i.e. `07:57`). This makes me suspect that these logs do not go back far enough. Do you have earlier logs? ---
**Author:** ![kley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kley/32/43533_2.png) [@kley](https://discuss.elastic.co/u/kley)\ **Post date:** [October 24, 2019, 11:24am UTC](https://discuss.elastic.co/t/does-elasticsearch-automatically-close-replicated-indices/205019/11 "2019-10-24T11:24:00Z")
Interestingly, the logs from the day before dont say anything at all - except some license warning. ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/f/9fe242538154fbb426ac231953c2137a8ae94f71.png) So, No logs before that ☹ ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [October 24, 2019, 12:36pm UTC](https://discuss.elastic.co/t/does-elasticsearch-automatically-close-replicated-indices/205019/12 "2019-10-24T12:36:32Z")
Another possible string that might be logged is `recovering as closed` after a full cluster restart, but in any case an index becoming closed is something that results in log messages. It's also not something that happens automatically to CCR followers, so we think it must have been an external influence which closed these indices. ---
**Author:** ![kley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kley/32/43533_2.png) [@kley](https://discuss.elastic.co/u/kley)\ **Post date:** [October 24, 2019, 12:43pm UTC](https://discuss.elastic.co/t/does-elasticsearch-automatically-close-replicated-indices/205019/13 "2019-10-24T12:43:59Z")
I'll monitor that cluster more closely in the future - if this happens again without any logs, I'll reply in this forum post again - hopefully with more info. Thank you so far David! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 21, 2019, 12:44pm UTC](https://discuss.elastic.co/t/does-elasticsearch-automatically-close-replicated-indices/205019/14 "2019-11-21T12:44:00Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.