.kibana index failed

Elastic Stack Kibana
1

hi,

could someone please advise on this.

2022-10-19T04:16:31.786+00:00][INFO ][savedobjects-service] Waiting until all Elasticsearch nodes are compatible with Kibana before starting saved objects migrations...
[2022-10-19T04:16:31.787+00:00][INFO ][savedobjects-service] Starting saved objects migrations
[2022-10-19T04:16:32.398+00:00][INFO ][savedobjects-service] [.kibana_task_manager] INIT -> OUTDATED_DOCUMENTS_SEARCH_OPEN_PIT. took: 203ms.
[2022-10-19T04:16:32.405+00:00][INFO ][savedobjects-service] [.kibana] INIT -> OUTDATED_DOCUMENTS_SEARCH_OPEN_PIT. took: 213ms.
[2022-10-19T04:16:32.484+00:00][INFO ][savedobjects-service] [.kibana_task_manager] OUTDATED_DOCUMENTS_SEARCH_OPEN_PIT -> OUTDATED_DOCUMENTS_SEARCH_READ. took: 86ms.
[2022-10-19T04:16:32.490+00:00][ERROR][savedobjects-service] [.kibana] Unexpected Elasticsearch ResponseError: statusCode: 500, method: POST, url: /.kibana_8.2.0_001/_pit?keep_alive=10m error: [search_phase_execution_exception]: all shards failed,
[2022-10-19T04:16:32.490+00:00][FATAL][root] Error: Unable to complete saved object migrations for the [.kibana] index. Please check the health of your Elasticsearch cluster and try again. Unexpected Elasticsearch ResponseError: statusCode: 500, method: POST, url: /.kibana_8.2.0_001/_pit?keep_alive=10m error: [search_phase_execution_exception]: all shards failed,
    at migrationStateActionMachine (/usr/share/kibana/src/core/server/saved_objects/migrations/migrations_state_action_machine.js:164:13)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

Thanks!

2

What's the status of Elasticsearch - red/green/yellow?

3

yellow.

4

we have two data centres, one DC is up and another is down because of index failed. is there a way we can create it .kibana index and bring that DC up ?

5

I am not sure what that means sorry.

What is the output from GET _cat/indices/.kibana?v?

6 
health status index             uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   .kibana_8.2.0_001 yLCqbJAGTF-VqLQl9D0mZA   1   1        549           14      2.5mb          2.5mb
7

Ok it's weird that it's complaining. Can you try restarting Kibana and see if it works for you?

8

yeah, I already did couple of time, but the same issue :expressionless:

9

It was failing because we have two cluster, the cluster-1 has joined the cluster-2 but due to different uuid it was showing the index in yellow and kibana shard failed.

Solution was to delete all the content from ~/var/lib/data-node/ and master-node from cluster-1 ( it will cause the data loss, since I had the new setup so I was okay) restart the nodes and it will join the cluster-2.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.