Reindex getting failed in kibana 6.8.13

Elastic Stack Kibana
1

Hi All

I have previous Data of ES v5.0.0 and i upgrade ES v5.0.0 to 6.8.13 after that i start reindex my cluster in kibana it got fail reason dont know..

I want ES to run on v7.6.3. For that i start with ES v6.8.13

My ES and Kibana both are having same version and i deploy both as a docker compose yaml file.

Here is my ES logs

[2021-03-08T13:50:54,676][WARN ][o.e.d.r.a.a.i.RestPutMappingAction] [Wke9CLL] [types removal] The parameter include_type_name should be explicitly specified in create index requests to prepare for 7.0. In 7.0 include_type_name will default to 'false', and requests are expected to omit the type name in mapping definitions.
[2021-03-08T13:50:54,688][INFO ][o.e.c.m.MetaDataCreateIndexService] [Wke9CLL] [reindexed-v6-fx-accounts] creating index, cause [api], templates [], shards [5]/[1], mapping
ue, index=.kibana_task_manager, type=_doc}
org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:296) [elasticsearch-6.8.13.jar:6.8.13]

Here is the Kibana Logs

est Timeout after 30000ms"}
{"type":"response","@timestamp":"2021-03-08T15:29:04Z","tags":[],"pid":1,"method":"put","statusCode":500,"req":{"url":"/api/upgrade_assistant/telemetry/ui_reindex","method":"put","headers":{"host":"192.168.0.13:5601","connection":"keep-alive","content-length":"14","kbn-version":"6.8.13","user-agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.66 Safari/537.36","content-type":"application/json","accept":"*/*","origin":"http://192.168.0.13:5601","referer":"http://192.168.0.13:5601/app/kibana","accept-encoding":"gzip, deflate","accept-language":"en-GB,en-US;q=0.9,en;q=0.8"},"remoteAddress":"10.0.0.2","userAgent":"10.0.0.2","referer":"http://192.168.0.13:5601/app/kibana"},"res":{"statusCode":500,"responseTime":30088,"contentLength":9},"message":"PUT /api/upgrade_assistant/telemetry/ui_reindex 500 30088ms - 9.0B"}
"type":"error","@timestamp":"2021-03-08T15:29:04Z","tags":200,"responseTime":7,"contentLength":9},"message":"GET /api/upgrade_assistant/reindex/fx-activity 200 7ms - 9.0B"}
{"type":"error","@timestamp":"2021-03-08T15:29:04Z","tags":[],"pid":1,"level":"error","error":{"message":"Request Timeout after 30000ms","name":"Error","stack":"Error: Request Timeout after 30000ms\n    at /usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:355:15\n    at Timeout.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:384:7)\n    at ontimeout (timers.js:436:11)\n    at tryOnTimeout (timers.js:300:5)\n    at listOnTimeout (timers.js:263:5)\n    at Timer.processTimers (timers.js:223:10)"},"url":{"protocol":null,"slashes":null,"auth":null,"host":null,"port":null,"hostname":null,"hash":null,"search":null,"query":{},"pathname":"/api/upgrade_assistant/telemetry/ui_reindex","path":"/api/upgrade_assistant/telemetry/ui_reindex","href":"/api/upgrade_assistant/telemetry/ui_reindex"},"message":"Request Timeout after 30000ms"}

This is Kibana URL
Reindexing process
Setting old index to read-only

Creating new index

There was an error
[resource_already_exists_exception] index [reindexed-v6-fx-activity/f6mfF5W7Sb2oLzsQF9yrFw] already exists, with { index_uuid="f6mfF5W7Sb2oLzsQF9yrFw" & index="reindexed-v6-fx-activity" } :: {"path":"/reindexed-v6-fx-activity","query":{},"body":"{\"settings\":{\"index.number_of_replicas\":\"1\",\"index.number_of_shards\":\"5\",\"index.refresh_interval\":\"1s\",\"index.store.type\":\"fs\"},\"mappings\":{\"activity\":{\"properties\":{\"activity\":{\"type\":\"text\",\"fields\":{\"keyword\":{\"type\":\"keyword\",\"ignore_above\":256}}},\"auth\":{\"type\":\"text\",\"fields\":{\"keyword\":{\"type\":\"keyword\",\"ignore_above\":256}}},\"autoCodeGeneratorCategory\":{\"type\":\"text\",\"fields\":{\"keyword\":{\"type\":\"keyword\",\"ignore_above\":256}}},\"autoCodeGeneratorId\":{\"type\":\"text\",\"fields\":{\"keyword\":
{\"type\":\"keyword\",\"ignore_above\":256}}},\"runId\":{\"type\":\"text\",\"fields\":{\"keyword\":{\"type\":\"keyword\",\"ignore_above\":256}}},\"status\":{\"type\":\"text\",\"fields\":{\"keyword\":{\"type\":\"keyword\",\"ignore_above\":256}}},\"testCaseNumber\":{\"type\":\"text\",\"fields\":{\"keyword\":{\"type\":\"keyword\",\"ignore_above\":256}}},\"testSuiteId\":{\"type\":\"text\",\"fields\":{\"keyword\":{\"type\":\"keyword\",\"ignore_above\":256}}},\"validations\":{\"type\":\"long\"}}}}}","statusCode":400,"response":"{\"error\":{\"root_cause\":[{\"type\":\"resource_already_exists_exception\",\"reason\":\"index [reindexed-v6-fx-activity/f6mfF5W7Sb2oLzsQF9yrFw] already exists\",\"index_uuid\":\"f6mfF5W7Sb2oLzsQF9yrFw\",\"index\":\"reindexed-v6-fx-activity\"}],\"type\":\"resource_already_exists_exception\",\"reason\":\"index [reindexed-v6-fx-activity/f6mfF5W7Sb2oLzsQF9yrFw] already exists\",\"index_uuid\":\"f6mfF5W7Sb2oLzsQF9yrFw\",\"index\":\"reindexed-v6-fx-activity\"},\"status\":400}"}

3.Reindexing documents

  1. Swapping original index with alias
2

What is the output from _cat/indices/.kibana*?v ?
To identify the root cause of the upgrade failure we would need to see the Kibana logs at the time when Kibana was upgraded (sharing the logs after restarting Kibana won't be useful as once a migration has failed, Kibana won't retry the migration until the migration lock has been removed).

Also follow this document - For instructions to remove a migration lock to allow Kibana to retry the migration seehttps://www.elastic.co/guide/en/kibana/current/upgrade-migrations.html

hope this helps
Rashmi

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.