# Error registering Kibana Privileges with Elasticsearch for kibana-.kibana: \[exception\] Trying to create too many scroll contexts **URL:** \ **Category:** Kibana\ **Created:** [November 30, 2020, 7:15pm UTC](https://discuss.elastic.co/t/error-registering-kibana-privileges-with-elasticsearch-for-kibana-kibana-exception-trying-to-create-too-many-scroll-contexts/257117 "2020-11-30T19:15:14Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![libbkmz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/libbkmz/32/79956_2.png) [@libbkmz](https://discuss.elastic.co/u/libbkmz)\ **Post date:** [November 30, 2020, 7:15pm UTC](https://discuss.elastic.co/t/error-registering-kibana-privileges-with-elasticsearch-for-kibana-kibana-exception-trying-to-create-too-many-scroll-contexts/257117/1 "2020-11-30T19:15:14Z")
Hello, I'm running many clusters of ElasticSearch with Kibana, we have version 7.6.2, this is version info: ```auto { "name" : "CLUSTER_NAME...", "cluster_name" : "CLUSTER_NAME", "cluster_uuid" : "4zR...", "version" : { "number" : "7.6.2", "build_flavor" : "default", "build_type" : "rpm", "build_hash" : "ef4...", "build_date" : "2020-03-26T06:34:37.794943Z", "build_snapshot" : false, "lucene_version" : "8.4.0", "minimum_wire_compatibility_version" : "6.8.0", "minimum_index_compatibility_version" : "6.0.0-beta1" }, "tagline" : "You Know, for Search" } ``` Here are some logs from kibana.log: ```auto {"type":"log","@timestamp":"2020-11-30T19:00:50Z","tags":["reporting","error"],"pid":17962,"message":"ErrorEvent {\n target:\n WebSocket {\n _events:\n [Object: null prototype] { open: [Function], error: [Function] },\n _eventsCount: 2,\n _maxListeners: undefined,\n readyState: 3,\n protocol: '',\n _binaryType: 'nodebuffer',\n _closeFrameReceived: false,\n _closeFrameSent: false,\n _closeMessage: '',\n _closeTimer: null,\n _closeCode: 1006,\n _extensions: {},\n _receiver: null,\n _sender: null,\n _socket: null,\n _isServer: false,\n _redirects: 0,\n url:\n 'ws://127.0.0.1:44141/devtools/browser/58fb58e3-4983-4de0-bd3b-47ad6dd47560',\n _req: null },\n type: 'error',\n message: 'connect ECONNREFUSED 127.0.0.1:44141',\n error:\n { Error: connect ECONNREFUSED 127.0.0.1:44141\n at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1107:14)\n errno: 'ECONNREFUSED',\n code: 'ECONNREFUSED',\n syscall: 'connect',\n address: '127.0.0.1',\n port: 44141 } }"} {"type":"log","@timestamp":"2020-11-30T19:00:50Z","tags":["reporting","warning"],"pid":17962,"message":"See Chromium's log output at \"/var/lib/kibana/headless_shell-linux/chrome_debug.log\""} {"type":"log","@timestamp":"2020-11-30T19:00:50Z","tags":["error","plugins","security","authorization"],"pid":17962,"message":"Error registering Kibana Privileges with Elasticsearch for kibana-.kibana: [exception] Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting."} {"type":"log","@timestamp":"2020-11-30T19:00:50Z","tags":["status","plugin:security@7.6.2","error"],"pid":17962,"state":"red","message":"Status changed from green to red - [exception] Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting.","prevState":"green","prevMsg":"Ready"} {"type":"log","@timestamp":"2020-11-30T19:00:50Z","tags":["error","plugins","security","authorization"],"pid":17962,"message":"Error registering Kibana Privileges with Elasticsearch for kibana-.kibana: [exception] Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting."} {"type":"log","@timestamp":"2020-11-30T19:00:52Z","tags":["error","plugins","taskManager","taskManager"],"pid":17962,"message":"Failed to poll for work: [exception] Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting. :: {\"path\":\"/.kibana_task_manager/_update_by_query\",\"query\":{\"ignore_unavailable\":true,\"refresh\":true,\"max_docs\":10,\"conflicts\":\"proceed\"},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"must\\\":[{\\\"term\\\":{\\\"type\\\":\\\"task\\\"}},{\\\"bool\\\":{\\\"must\\\":[{\\\"bool\\\":{\\\"should\\\":[{\\\"bool\\\":{\\\"must\\\":[{\\\"term\\\":{\\\"task.status\\\":\\\"idle\\\"}},{\\\"range\\\":{\\\"task.runAt\\\":{\\\"lte\\\":\\\"now\\\"}}}]}},{\\\"bool\\\":{\\\"must\\\":[{\\\"bool\\\":{\\\"should\\\":[{\\\"term\\\":{\\\"task.status\\\":\\\"running\\\"}},{\\\"term\\\":{\\\"task.status\\\":\\\"claiming\\\"}}]}},{\\\"range\\\":{\\\"task.retryAt\\\":{\\\"lte\\\":\\\"now\\\"}}}]}}]}},{\\\"bool\\\":{\\\"should\\\":[{\\\"exists\\\":{\\\"field\\\":\\\"task.schedule\\\"}},{\\\"bool\\\":{\\\"must\\\":[{\\\"term\\\":{\\\"task.taskType\\\":\\\"actions:.server-log\\\"}},{\\\"range\\\":{\\\"task.attempts\\\":{\\\"lt\\\":1}}}]}},{\\\"bool\\\":{\\\"must\\\":[{\\\"term\\\":{\\\"task.taskType\\\":\\\"actions:.slack\\\"}},{\\\"range\\\":{\\\"task.attempts\\\":{\\\"lt\\\":1}}}]}},{\\\"bool\\\":{\\\"must\\\":[{\\\"term\\\":{\\\"task.taskType\\\":\\\"actions:.email\\\"}},{\\\"range\\\":{\\\"task.attempts\\\":{\\\"lt\\\":1}}}]}},{\\\"bool\\\":{\\\"must\\\":[{\\\"term\\\":{\\\"task.taskType\\\":\\\"actions:.index\\\"}},{\\\"range\\\":{\\\"task.attempts\\\":{\\\"lt\\\":1}}}]}},{\\\"bool\\\":{\\\"must\\\":[{\\\"term\\\":{\\\"task.taskType\\\":\\\"actions:.pagerduty\\\"}},{\\\"range\\\":{\\\"task.attempts\\\":{\\\"lt\\\":1}}}]}},{\\\"bool\\\":{\\\"must\\\":[{\\\"term\\\":{\\\"task.taskType\\\":\\\"actions:.webhook\\\"}},{\\\"range\\\":{\\\"task.attempts\\\":{\\\"lt\\\":1}}}]}},{\\\"bool\\\":{\\\"must\\\":[{\\\"term\\\":{\\\"task.taskType\\\":\\\"alerting:siem.signals\\\"}},{\\\"range\\\":{\\\"task.attempts\\\":{\\\"lt\\\":3}}}]}},{\\\"bool\\\":{\\\"must\\\":[{\\\"term\\\":{\\\"task.taskType\\\":\\\"vis_telemetry\\\"}},{\\\"range\\\":{\\\"task.attempts\\\":{\\\"lt\\\":3}}}]}},{\\\"bool\\\":{\\\"must\\\":[{\\\"term\\\":{\\\"task.taskType\\\":\\\"lens_telemetry\\\"}},{\\\"range\\\":{\\\"task.attempts\\\":{\\\"lt\\\":3}}}]}}]}}]}}]}},\\\"sort\\\":{\\\"_script\\\":{\\\"type\\\":\\\"number\\\",\\\"order\\\":\\\"asc\\\",\\\"script\\\":{\\\"lang\\\":\\\"painless\\\",\\\"source\\\":\\\"\\\\nif (doc['task.retryAt'].size()!=0) {\\\\n return doc['task.retryAt'].value.toInstant().toEpochMilli();\\\\n}\\\\nif (doc['task.runAt'].size()!=0) {\\\\n return doc['task.runAt'].value.toInstant().toEpochMilli();\\\\n}\\\\n \\\"}}},\\\"seq_no_primary_term\\\":true,\\\"script\\\":{\\\"source\\\":\\\"ctx._source.task.ownerId=params.ownerId; ctx._source.task.status=params.status; ctx._source.task.retryAt=params.retryAt;\\\",\\\"lang\\\":\\\"painless\\\",\\\"params\\\":{\\\"ownerId\\\":\\\"kibana:86dce749-943a-49bb-97ba-e89b81c0a007\\\",\\\"status\\\":\\\"claiming\\\",\\\"retryAt\\\":\\\"2020-11-30T19:01:22.921Z\\\"}}}\",\"statusCode\":500,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"exception\\\",\\\"reason\\\":\\\"Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting.\\\"}],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[{\\\"shard\\\":0,\\\"index\\\":\\\".kibana_task_manager_1\\\",\\\"node\\\":\\\"1yzPo81-Rl2c2tg1bqhJmg\\\",\\\"reason\\\":{\\\"type\\\":\\\"exception\\\",\\\"reason\\\":\\\"Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting.\\\"}}]},\\\"status\\\":500}\"}"} ``` I've even tried to increase the number of available scrolls, you can see it by this log line: ```auto {"type":"log","@timestamp":"2020-11-30T19:03:05Z","tags":["error","plugins","security","authorization"],"pid":17962,"message":"Error registering Kibana Privileges with Elasticsearch for kibana-.kibana: [exception] Trying to create too many scroll contexts. Must be less than or equal to: [500000]. This limit can be set by changing the [search.max_open_scroll_context] setting."} ``` I have tried to delete `.kibana_1` index and `.kibana_task_manager_1`. But nothing helps. Also, I'm getting this errors when try to reproduce the error simply by reindex task: ```auto ~> curl -sS localhost:9200/_reindex -H "Content-type: application/json" -d '{"source": {"index":".kibana_1"}, "dest":{"index":".backup_kibana_1"}}' | jq . { "error": { "root_cause": [ { "type": "exception", "reason": "Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting." } ], "type": "search_phase_execution_exception", "reason": "all shards failed", "phase": "query", "grouped": true, "failed_shards": [ { "shard": 0, "index": ".kibana_1", "node": "hVa...", "reason": { "type": "exception", "reason": "Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting." } } ] }, "status": 500 } ~> curl -sS localhost:9200/_reindex -H "Content-type: application/json" -d '{"source": {"index":".kibana_task_manager_1"}, "dest":{"index":".backup_kibana_task_manger_1"}}' | jq . { "error": { "root_cause": [ { "type": "exception", "reason": "Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting." } ], "type": "search_phase_execution_exception", "reason": "all shards failed", "phase": "query", "grouped": true, "failed_shards": [ { "shard": 0, "index": ".kibana_task_manager_1", "node": "1yz...", "reason": { "type": "exception", "reason": "Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting." } } ] }, "status": 500 } ``` According to server stats, nobody uses scroll API, only kibana\_task\_manager is getting higher and higher... ---
**Author:** ![styks90](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/styks90/32/59205_2.png) [@styks90](https://discuss.elastic.co/u/styks90)\ **Post date:** [December 1, 2020, 9:27am UTC](https://discuss.elastic.co/t/error-registering-kibana-privileges-with-elasticsearch-for-kibana-kibana-exception-trying-to-create-too-many-scroll-contexts/257117/2 "2020-12-01T09:27:08Z")
By coincidence or not, we started seeing the same issue and error messages yesterday. Version info: ``` { "name" : "NODENAME", "cluster_name" : "CLUSTERNAME", "cluster_uuid" : "CLUSTERID", "version" : { "number" : "7.6.2", "build_flavor" : "default", "build_type" : "docker", "build_hash" : "ef48eb35cf30adf4db14086e8aabd07ef6fb113f", "build_date" : "2020-03-26T06:34:37.794943Z", "build_snapshot" : false, "lucene_version" : "8.4.0", "minimum_wire_compatibility_version" : "6.8.0", "minimum_index_compatibility_version" : "6.0.0-beta1" }, "tagline" : "You Know, for Search" } ``` After a restart of Elasticsearch, the node stats also look completely fine: ``` "indices" : { "search" : { "open_contexts" : 0, "query_total" : 4703044, "query_time_in_millis" : 138223, "query_current" : 0, "fetch_total" : 930, "fetch_time_in_millis" : 2029, "fetch_current" : 0, "scroll_total" : 437, "scroll_time_in_millis" : 3205, "scroll_current" : 0, "suggest_total" : 0, "suggest_time_in_millis" : 0, "suggest_current" : 0 } } ``` The issue seems to be caused by the security plugin, which crashes our Kibana instance: ![image](https://us1.discourse-cdn.com/elastic/original/3X/d/c/dcbd93e7fc07f403fa01f0c1f02e1ffa4ea7a4e1.png) ---
**Author:** ![libbkmz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/libbkmz/32/79956_2.png) [@libbkmz](https://discuss.elastic.co/u/libbkmz)\ **Post date:** [December 1, 2020, 3:33pm UTC](https://discuss.elastic.co/t/error-registering-kibana-privileges-with-elasticsearch-for-kibana-kibana-exception-trying-to-create-too-many-scroll-contexts/257117/3 "2020-12-01T15:33:46Z")
My complaints: - There is no way to find any information about current scrolls in the system. No \_cat API for that - Even after increasing number of scrolls to 500k it still saying that it can't create a new scroll: > Error registering Kibana Privileges with Elasticsearch for kibana-.kibana: [exception] Trying to create too many scroll contexts. Must be less than or equal to: [**500000**]. ---
**Author:** ![libbkmz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/libbkmz/32/79956_2.png) [@libbkmz](https://discuss.elastic.co/u/libbkmz)\ **Post date:** [December 8, 2020, 3:09pm UTC](https://discuss.elastic.co/t/error-registering-kibana-privileges-with-elasticsearch-for-kibana-kibana-exception-trying-to-create-too-many-scroll-contexts/257117/4 "2020-12-08T15:09:13Z")
It seems like that these steps can fix it somehow: - stop kibana - Disable xpack.security in kibana.yaml - curl -sS localhost:9200/\_search/scroll?scroll\_id=\_all -XDELETE - DELETE ALL KIBANA INDICES !!!!11 (.kibana\_1, .kibana\_task\_manager\_1, .security\_7, .apm-agent-configuration) - start kibana Please be aware that it will delete everything you have in you kibana, all index patters, all charts, all dashboards ---
**Author:** ![styks90](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/styks90/32/59205_2.png) [@styks90](https://discuss.elastic.co/u/styks90)\ **Post date:** [December 15, 2020, 8:31am UTC](https://discuss.elastic.co/t/error-registering-kibana-privileges-with-elasticsearch-for-kibana-kibana-exception-trying-to-create-too-many-scroll-contexts/257117/5 "2020-12-15T08:31:32Z")
If possible, upgrading Kibana also seems to work. We've upgraded to 7.9.2, without touching configuration and system indices, and do not see the issue anymore. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 12, 2021, 8:31am UTC](https://discuss.elastic.co/t/error-registering-kibana-privileges-with-elasticsearch-for-kibana-kibana-exception-trying-to-create-too-many-scroll-contexts/257117/6 "2021-01-12T08:31:36Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.