# ELK-log full with errors **URL:** \ **Category:** Elasticsearch\ **Tags:** elastic-stack-security\ **Created:** [August 31, 2020, 8:57pm UTC](https://discuss.elastic.co/t/elk-log-full-with-errors/247017 "2020-08-31T20:57:57Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![NogNeetMachinaal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nogneetmachinaal/32/58893_2.png) [@NogNeetMachinaal](https://discuss.elastic.co/u/NogNeetMachinaal)\ **Post date:** [August 31, 2020, 8:57pm UTC](https://discuss.elastic.co/t/elk-log-full-with-errors/247017/1 "2020-08-31T20:57:57Z")
Team - The log of ElasticSearch is full with errors (see below). Most likely, because of this the system is not working as expected. Any idea what is happening and how to fix this? It is version 7.9 for Elastic and Kibana. Beats can be various versions - 7.6 to 7.9. # Thank you - Will ```auto [2020-08-31T22:31:16,679][WARN][o.e.i.s.IndexShard] [elk.itv.lan] [auditbeat-7.9.0-2020.08.28-000001][0] onPreFetchPhase listener [org.elasticsearch.xpack.security.authz.SecuritySearchOperationListener@36669906] failed org.elasticsearch.ElasticsearchSecurityException: [[Bgw3RnQBQYQGFTWYgG3G][1408887]] expected scroll indices access control [IndicesAccessControl{granted=true, indexPermissions={auditbeat-7.6.2-2020.08.05-000011=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, auditbeat-7.7.0-2020.08.04-000008=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, auditbeat-7.6.2-2020.08.20-000012=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, auditbeat-7.8.0-2020.08.18-000005=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, auditbeat-7.7.1-2020.08.02-000005=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, auditbeat-7.7.0-2020.08.19-000009=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, auditbeat-7.6.2=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, auditbeat-7.7.1=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, auditbeat-7.8.0=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, auditbeat-7.7.0=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, auditbeat-7.7.1-2020.08.17-000006=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, auditbeat-7.9.0=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, auditbeat-7.8.0-2020.08.03-000004=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, auditbeat-7.9.0-2020.08.28-000001=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}}}] but found [IndicesAccessControl{granted=true, indexPermissions={auditbeat-7.6.2-2020.08.05-000011=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, auditbeat-7.7.0-2020.08.04-000008=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, auditbeat-7.6.2-2020.08.20-000012=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, auditbeat-7.8.0-2020.08.18-000005=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, auditbeat-7.7.1-2020.08.02-000005=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, auditbeat-7.7.0-2020.08.19-000009=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, auditbeat-7.6.2=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, auditbeat-7.7.1=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, auditbeat-7.8.0=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, auditbeat-7.7.0=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, auditbeat-7.7.1-2020.08.17-000006=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, auditbeat-7.9.0=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, auditbeat-7.8.0-2020.08.03-000004=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, auditbeat-7.9.0-2020.08.28-000001=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}}}] in thread context at org.elasticsearch.xpack.security.authz.SecuritySearchOperationListener.ensureIndicesAccessControlForScrollThreadContext(SecuritySearchOperationListener.java:111) ~[?:?] at org.elasticsearch.xpack.security.authz.SecuritySearchOperationListener.onPreFetchPhase(SecuritySearchOperationListener.java:95) ~[?:?] at org.elasticsearch.index.shard.SearchOperationListener$CompositeListener.onPreFetchPhase(SearchOperationListener.java:166) [elasticsearch-7.9.0.jar:7.9.0] at org.elasticsearch.search.SearchService$SearchOperationListenerExecutor.(SearchService.java:1272) [elasticsearch-7.9.0.jar:7.9.0] at org.elasticsearch.search.SearchService.lambda$executeFetchPhase$4(SearchService.java:584) [elasticsearch-7.9.0.jar:7.9.0] at org.elasticsearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:58) [elasticsearch-7.9.0.jar:7.9.0] at org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:73) [elasticsearch-7.9.0.jar:7.9.0] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.9.0.jar:7.9.0] at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:44) [elasticsearch-7.9.0.jar:7.9.0] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:710) [elasticsearch-7.9.0.jar:7.9.0] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.9.0.jar:7.9.0] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?] at java.lang.Thread.run(Thread.java:832) [?:?] ``` ---
**Author:** ![MakoWish](https://avatars.discourse-cdn.com/v4/letter/m/e56c9b/32.png) [@MakoWish](https://discuss.elastic.co/u/MakoWish)\ **Post date:** [September 2, 2020, 2:22pm UTC](https://discuss.elastic.co/t/elk-log-full-with-errors/247017/2 "2020-09-02T14:22:29Z")
I am having the same issue. Elastic and Kibana are also on 7.9. Most Beats should be 7.9, but there may be a few stragglers that have not yet updated and may be on 7.8.0. Your issue keeps calling out Auditbeat indices, but mine is calling out Heartbeat indices. ```auto [2020-09-02T07:15:06,083][WARN][o.e.i.s.IndexShard] [ElasticData01] [heartbeat-7.9.0-2020.08.20-000001][0] onPreFetchPhase listener [org.elasticsearch.xpack.security.authz.SecuritySearchOperationListener@256cfac9] failed org.elasticsearch.ElasticsearchSecurityException: [[Y9orT3QBDv1cS7Tg0uBN][2006027]] expected scroll indices access control [IndicesAccessControl{granted=true, indexPermissions={heartbeat-7.9.0-2020.08.20-000001=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1120a, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, heartbeat-7.7.0-2020.06.21-000002=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1120a, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, heartbeat-7.8.0-2020.07.29-000002=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1120a, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, heartbeat-7.8.0-2020.08.28-000003=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1120a, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, heartbeat-7.8.0=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1120a, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, heartbeat-7.8.0-2020.06.29-000001=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1120a, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, heartbeat-7.9.0=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1120a, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, heartbeat-7.7.0=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1120a, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, heartbeat-7.7.0-2020.05.22-000001=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1120a, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}}}] but found [IndicesAccessControl{granted=true, indexPermissions={heartbeat-7.9.0-2020.08.20-000001=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1120a, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, heartbeat-7.7.0-2020.06.21-000002=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1120a, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, heartbeat-7.8.0-2020.07.29-000002=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1120a, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, heartbeat-7.8.0-2020.08.28-000003=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1120a, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, heartbeat-7.8.0=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1120a, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, heartbeat-7.8.0-2020.06.29-000001=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1120a, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, heartbeat-7.9.0=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1120a, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, heartbeat-7.7.0=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1120a, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}, heartbeat-7.7.0-2020.05.22-000001=IndexAccessControl{granted=true, fieldPermissions=org.elasticsearch.xpack.core.security.authz.permission.FieldPermissions@1120a, documentPermissions=DocumentPermissions [queries=null, scopedByQueries=null]}}}] in thread context at org.elasticsearch.xpack.security.authz.SecuritySearchOperationListener.ensureIndicesAccessControlForScrollThreadContext(SecuritySearchOperationListener.java:111) ~[?:?] at org.elasticsearch.xpack.security.authz.SecuritySearchOperationListener.onPreFetchPhase(SecuritySearchOperationListener.java:95) ~[?:?] at org.elasticsearch.index.shard.SearchOperationListener$CompositeListener.onPreFetchPhase(SearchOperationListener.java:166) [elasticsearch-7.9.0.jar:7.9.0] at org.elasticsearch.search.SearchService$SearchOperationListenerExecutor.(SearchService.java:1272) [elasticsearch-7.9.0.jar:7.9.0] at org.elasticsearch.search.SearchService.lambda$executeFetchPhase$4(SearchService.java:584) [elasticsearch-7.9.0.jar:7.9.0] at org.elasticsearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:58) [elasticsearch-7.9.0.jar:7.9.0] at org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:73) [elasticsearch-7.9.0.jar:7.9.0] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.9.0.jar:7.9.0] at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:44) [elasticsearch-7.9.0.jar:7.9.0] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:710) [elasticsearch-7.9.0.jar:7.9.0] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.9.0.jar:7.9.0] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?] at java.lang.Thread.run(Thread.java:832) [?:?] ``` ---
**Author:** ![Albert\_Zaharovits](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/albert_zaharovits/32/24390_2.png) [@Albert\_Zaharovits](https://discuss.elastic.co/u/Albert_Zaharovits)\ **Post date:** [September 2, 2020, 3:24pm UTC](https://discuss.elastic.co/t/elk-log-full-with-errors/247017/3 "2020-09-02T15:24:58Z")
This is a spurious log warn message, it is not indicative of any problems. This is fixed in 7.9.1 . Apologies for the inconvenience. ---
**Author:** ![MakoWish](https://avatars.discourse-cdn.com/v4/letter/m/e56c9b/32.png) [@MakoWish](https://discuss.elastic.co/u/MakoWish)\ **Post date:** [September 2, 2020, 3:46pm UTC](https://discuss.elastic.co/t/elk-log-full-with-errors/247017/4 "2020-09-02T15:46:41Z")
Thank you, @Albert_Zaharovits. Glad to hear it is not an actual issue. Do you know when 7.9.1 will reach GA? ---
**Author:** ![NogNeetMachinaal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nogneetmachinaal/32/58893_2.png) [@NogNeetMachinaal](https://discuss.elastic.co/u/NogNeetMachinaal)\ **Post date:** [September 2, 2020, 4:43pm UTC](https://discuss.elastic.co/t/elk-log-full-with-errors/247017/5 "2020-09-02T16:43:40Z")
Perhaps true - but in that case it prevents troubleshooting a real issue: no Elastic and no Kibana =\> yes - the processes are running. But there is no login for Kibana. And the Beats-services can not connect to ElasticSearch - connection refused. All this started with version 7.9 =\> until the 7.8-, most of the things are working as expected. Any suggestions? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 30, 2020, 4:44pm UTC](https://discuss.elastic.co/t/elk-log-full-with-errors/247017/6 "2020-09-30T16:44:02Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.