Error: Unable to complete saved object migrations for the [.kibana_task_manager] index

Elastic Stack Elasticsearch
1

Hi Elasticsearch Team, earlier I tried to close some indexes while performing Restore, and the Kibana turns down and not accessible. Can I have your help on this?

{"type":"log","@timestamp":"2022-07-14T16:36:17+10:00","tags":["info","savedobjects-service"],"pid":59818,"message":"Waiting until all Elasticsearch nodes are compatible with Kibana before starting saved objects migrations..."}
{"type":"log","@timestamp":"2022-07-14T16:36:17+10:00","tags":["info","savedobjects-service"],"pid":59818,"message":"Starting saved objects migrations"}
{"type":"log","@timestamp":"2022-07-14T16:36:19+10:00","tags":["info","savedobjects-service"],"pid":59818,"message":"[.kibana] INIT -> OUTDATED_DOCUMENTS_SEARCH_OPEN_PIT. took: 81ms."}
{"type":"log","@timestamp":"2022-07-14T16:36:19+10:00","tags":["info","savedobjects-service"],"pid":59818,"message":"[.kibana_task_manager] INIT -> OUTDATED_DOCUMENTS_SEARCH_OPEN_PIT. took: 82ms."}
{"type":"log","@timestamp":"2022-07-14T16:36:19+10:00","tags":["info","savedobjects-service"],"pid":59818,"message":"[.kibana] OUTDATED_DOCUMENTS_SEARCH_OPEN_PIT -> OUTDATED_DOCUMENTS_SEARCH_READ. took: 30ms."}
{"type":"log","@timestamp":"2022-07-14T16:36:19+10:00","tags":["info","savedobjects-service"],"pid":59818,"message":"[.kibana_task_manager] OUTDATED_DOCUMENTS_SEARCH_OPEN_PIT -> OUTDATED_DOCUMENTS_SEARCH_READ. took: 31ms."}
{"type":"log","@timestamp":"2022-07-14T16:36:19+10:00","tags":["info","savedobjects-service"],"pid":59818,"message":"[.kibana] OUTDATED_DOCUMENTS_SEARCH_READ -> OUTDATED_DOCUMENTS_SEARCH_CLOSE_PIT. took: 14ms."}
{"type":"log","@timestamp":"2022-07-14T16:36:19+10:00","tags":["info","savedobjects-service"],"pid":59818,"message":"[.kibana_task_manager] OUTDATED_DOCUMENTS_SEARCH_READ -> OUTDATED_DOCUMENTS_SEARCH_CLOSE_PIT. took: 13ms."}
{"type":"log","@timestamp":"2022-07-14T16:36:19+10:00","tags":["info","savedobjects-service"],"pid":59818,"message":"[.kibana] OUTDATED_DOCUMENTS_SEARCH_CLOSE_PIT -> UPDATE_TARGET_MAPPINGS. took: 5ms."}
{"type":"log","@timestamp":"2022-07-14T16:36:19+10:00","tags":["info","savedobjects-service"],"pid":59818,"message":"[.kibana_task_manager] OUTDATED_DOCUMENTS_SEARCH_CLOSE_PIT -> UPDATE_TARGET_MAPPINGS. took: 6ms."}
{"type":"log","@timestamp":"2022-07-14T16:36:19+10:00","tags":["info","savedobjects-service"],"pid":59818,"message":"[.kibana_task_manager] UPDATE_TARGET_MAPPINGS -> UPDATE_TARGET_MAPPINGS_WAIT_FOR_TASK. took: 17ms."}
{"type":"log","@timestamp":"2022-07-14T16:36:19+10:00","tags":["info","savedobjects-service"],"pid":59818,"message":"[.kibana] UPDATE_TARGET_MAPPINGS -> UPDATE_TARGET_MAPPINGS_WAIT_FOR_TASK. took: 46ms."}
{"type":"log","@timestamp":"2022-07-14T16:36:19+10:00","tags":["error","savedobjects-service"],"pid":59818,"message":"[.kibana_task_manager] Unexpected Elasticsearch ResponseError: statusCode: 400, method: GET, url: /_tasks/lknaevIIRmqBEWw_qYgDyg%3A24597983?wait_for_completion=true&timeout=60s error: [index_closed_exception]: closed,"}
{"type":"log","@timestamp":"2022-07-14T16:36:19+10:00","tags":["fatal","root"],"pid":59818,"message":"Error: Unable to complete saved object migrations for the [.kibana_task_manager] index. Please check the health of your Elasticsearch cluster and try again. Unexpected Elasticsearch ResponseError: statusCode: 400, method: GET, url: /_tasks/lknaevIIRmqBEWw_qYgDyg%3A24597983?wait_for_completion=true&timeout=60s error: [index_closed_exception]: closed,\n at migrationStateActionMachine (/opt/elasticsearch/elasticsearchkibana/dev/kibana-7.16.2-linux-x86_64/src/core/server/saved_objects/migrationsv2/migrations_state_action_machine.js:164:13)\n at processTicksAndRejections (node:internal/process/task_queues:96:5)\n at async Promise.all (index 1)\n at SavedObjectsService.start (/opt/elasticsearch/elasticsearchkibana/dev/kibana-7.16.2-linux-x86_64/src/core/server/saved_objects/saved_objects_service.js:181:9)\n at Server.start (/opt/elasticsearch/elasticsearchkibana/dev/kibana-7.16.2-linux-x86_64/src/core/server/server.js:330:31)\n at Root.start (/opt/elasticsearch/elasticsearchkibana/dev/kibana-7.16.2-linux-x86_64/src/core/server/root/index.js:69:14)\n at bootstrap (/opt/elasticsearch/elasticsearchkibana/dev/kibana-7.16.2-linux-x86_64/src/core/server/bootstrap.js:120:5)\n at Command. (/opt/elasticsearch/elasticsearchkibana/dev/kibana-7.16.2-linux-x86_64/src/cli/serve/serve.js:229:5)"}
{"type":"log","@timestamp":"2022-07-14T16:36:20+10:00","tags":["error","savedobjects-service"],"pid":59818,"message":"[.kibana] Unexpected Elasticsearch ResponseError: statusCode: 400, method: GET, url: /_tasks/lknaevIIRmqBEWw_qYgDyg%3A24597991?wait_for_completion=true&timeout=60s error: [index_closed_exception]: closed,"}
{"type":"log","@timestamp":"2022-07-14T16:36:21+10:00","tags":["info","plugins-system","standard"],"pid":59818,"message":"Stopping all plugins."}
{"type":"log","@timestamp":"2022-07-14T16:36:21+10:00","tags":["info","plugins","monitoring","monitoring","kibana-monitoring"],"pid":59818,"message":"Monitoring stats collection is stopped"}
{"type":"log","@timestamp":"2022-07-14T16:36:51+10:00","tags":["warning","plugins-system","standard"],"pid":59818,"message":""eventLog" plugin didn't stop in 30sec., move on to the next."}

FATAL Error: Unable to complete saved object migrations for the [.kibana_task_manager] index. Please check the health of your Elasticsearch cluster and try again. Unexpected Elasticsearch ResponseError: statusCode: 400, method: GET, url: /_tasks/lknaevIIRmqBEWw_qYgDyg%3A24597983?wait_for_completion=true&timeout=60s error: [index_closed_exception]: closed,

2

It seems the Kibana is Corrupted because of closing the kibana security. Is it possible to use POST /_all/_open ? If yes, how is it possible when the Kibana is not accessible?

3

Solved by re-opening indices using the command curl -u "credential" -XPOST localhost:9200/_all/_open

4

iam also facing same isuue, where can i run this command
curl -u "credential" -XPOST localhost:9200/_all/_open

5

u can run this on command prompt :slight_smile:

6

thanks. FYI if we run below command
curl -u "crdetial_of_es" -XPOST https://localhost:9200/_all/_open
we get following error
curl: (35) schannel: next InitializeSecurityContext failed: Unknown error (0x80092012) - The revocation function was unable to check revocation for the certif
so this can be avoided by following command
C:\Users\Administrator>curl -u "crdetial_of_es" -XPOST https://localhost:9200/_all/_open --ssl-no-revoke -L

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.