# Kibana cannot perform migrations due to validation error

**URL:** <https://discuss.elastic.co/t/kibana-cannot-perform-migrations-due-to-validation-error/278678>\
**Category:** Kibana\
**Created:** [July 14, 2021, 2:08pm UTC](https://discuss.elastic.co/t/kibana-cannot-perform-migrations-due-to-validation-error/278678 "2021-07-14T14:08:39Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![dbernard](https://avatars.discourse-cdn.com/v4/letter/d/d9b06d/32.png) [@dbernard](https://discuss.elastic.co/u/dbernard)\
**Post date:** [July 14, 2021, 2:08pm UTC](https://discuss.elastic.co/t/kibana-cannot-perform-migrations-due-to-validation-error/278678/1 "2021-07-14T14:08:39Z")

</div>

Kibana is unable to perform migrations and then reports it cannot connect to elastic search.  
elasticsearch is running locally.

Upgrading to 7.11.2 elasticsearch and kibana.

```auto
"type":"log","@timestamp":"2021-07-14T14:02:53+00:00","tags":["info","savedobjects-service"],"pid":830523,"message":"Starting saved objects migrations"}
{"type":"log","@timestamp":"2021-07-14T14:02:53+00:00","tags":["info","savedobjects-service"],"pid":830523,"message":"Creating index .kibana_task_manager_1."}
{"type":"log","@timestamp":"2021-07-14T14:02:53+00:00","tags":["warning","savedobjects-service"],"pid":830523,"message":"Unable to connect to Elasticsearch. Error: validation_exception"}
{"type":"log","@timestamp":"2021-07-14T14:02:53+00:00","tags":["fatal","root"],"pid":830523,"message":"ResponseError: validation_exception\n at onBody (/usr/share/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:333:23)\n at IncomingMessage.onEnd (/usr/share/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:260:11)\n at IncomingMessage.emit (events.js:327:22)\n at endReadableNT (internal/streams/readable.js:1327:12)\n at processTicksAndRejections (internal/process/task_queues.js:80:21) {\n meta: {\n body: { error: [Object], status: 400 },\n statusCode: 400,\n headers: {\n 'content-type': 'application/json; charset=UTF-8',\n 'content-length': '379'\n },\n meta: {\n context: null,\n request: [Object],\n name: 'elasticsearch-js',\n connection: [Object],\n attempts: 0,\n aborted: false\n }\n }\n}"}
{"type":"log","@timestamp":"2021-07-14T14:02:53+00:00","tags":["info","plugins-system"],"pid":830523,"message":"Stopping all plugins."}
{"type":"log","@timestamp":"2021-07-14T14:02:53+00:00","tags":["info","plugins","monitoring","monitoring","kibana-monitoring"],"pid":830523,"message":"Monitoring stats collection is stopped"}
{"type":"log","@timestamp":"2021-07-14T14:02:53+00:00","tags":["info","savedobjects-service"],"pid":830523,"message":"Creating index .kibana_2."}
{"type":"log","@timestamp":"2021-07-14T14:02:53+00:00","tags":["warning","savedobjects-service"],"pid":830523,"message":"Unable to connect to Elasticsearch. Error: validation_exception"}

```

---

<div class="post-metadata">

**Author:** ![cheiligers](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cheiligers/32/73114_2.png) [@cheiligers](https://discuss.elastic.co/u/cheiligers)\
**Post date:** [July 14, 2021, 7:12pm UTC](https://discuss.elastic.co/t/kibana-cannot-perform-migrations-due-to-validation-error/278678/2 "2021-07-14T19:12:08Z")

</div>

@dbernard please see [this discuss post](https://discuss.elastic.co/t/kibana-7-11-2-fails-to-start-post-upgrade/266926) that walks through some debugging steps. If that doesn't solve your issue, could you provide more verbose logs please?

---

<div class="post-metadata">

**Author:** ![dbernard](https://avatars.discourse-cdn.com/v4/letter/d/d9b06d/32.png) [@dbernard](https://discuss.elastic.co/u/dbernard)\
**Post date:** [July 15, 2021, 12:09am UTC](https://discuss.elastic.co/t/kibana-cannot-perform-migrations-due-to-validation-error/278678/3 "2021-07-15T00:09:01Z")

</div>

There are a bunch of start up errors I'll look at.  
kibana 6.2.4-1 installation was inadvertently upgraded to 7.10 then 7.12 by accident.

```auto
{"type":"log","@timestamp":"2021-07-15T00:19:26+00:00","tags":["warning","plugins","licensing"],"pid":1837423,"message":"License information could not be obtained from Elasticsearch due to Error: Cluster client cannot be used after it has been closed. error"}
{"type":"log","@timestamp":"2021-07-15T00:19:56+00:00","tags":["warning","plugins","licensing"],"pid":1837423,"message":"License information could not be obtained from Elasticsearch due to Error: Cluster client cannot be used after it has been closed. error"}
{"type":"log","@timestamp":"2021-07-15T00:20:26+00:00","tags":["warning","plugins","licensing"],"pid":1837423,"message":"License information could not be obtained from Elasticsearch due to Error: Cluster client cannot be used after it has been closed. error"}
{"type":"log","@timestamp":"2021-07-15T00:20:56+00:00","tags":["warning","plugins","licensing"],"pid":1837423,"message":"License information could not be obtained from Elasticsearch due to Error: Cluster client cannot be used after it has been closed. error"}
{"type":"log","@timestamp":"2021-07-15T00:21:10+00:00","tags":["info","plugins-system"],"pid":1837423,"message":"Stopping all plugins."}
{"type":"log","@timestamp":"2021-07-15T00:21:17+00:00","tags":["info","plugins-service"],"pid":1845073,"message":"Plugin \"visTypeXy\" is disabled."}
{"type":"log","@timestamp":"2021-07-15T00:21:17+00:00","tags":["warning","config","deprecation"],"pid":1845073,"message":"Setting [elasticsearch.username] to \"elastic\" is deprecated. You should use the \"kibana_system\" user instead."}
{"type":"log","@timestamp":"2021-07-15T00:21:17+00:00","tags":["warning","config","deprecation"],"pid":1845073,"message":"Config key [monitoring.cluster_alerts.email_notifications.email_address] will be required for email notifications to work in 8.0.\""}
{"type":"log","@timestamp":"2021-07-15T00:21:17+00:00","tags":["warning","config","deprecation"],"pid":1845073,"message":"Setting [monitoring.username] to \"elastic\" is deprecated. You should use the \"kibana_system\" user instead."}
{"type":"log","@timestamp":"2021-07-15T00:21:18+00:00","tags":["info","plugins-system"],"pid":1845073,"message":"Setting up [102] plugins: [taskManager,licensing,globalSearch,globalSearchProviders,code,usageCollection,xpackLegacy,telemetryCollectionManager,telemetry,telemetryCollectionXpack,kibanaUsageCollection,securityOss,newsfeed,mapsLegacy,kibanaLegacy,translations,share,legacyExport,embeddable,uiActionsEnhanced,expressions,charts,esUiShared,bfetch,data,home,observability,console,consoleExtensions,apmOss,searchprofiler,painlessLab,grokdebugger,management,indexPatternManagement,advancedSettings,fileUpload,savedObjects,visualizations,visTypeVislib,visTypeVega,visTypeTimeseries,visTypeTimeseriesEnhanced,visTypeTimelion,features,licenseManagement,dataEnhanced,watcher,canvas,visTypeTagcloud,visTypeTable,visTypeMetric,visTypeMarkdown,tileMap,regionMap,mapsOss,lensOss,inputControlVis,graph,timelion,dashboard,dashboardEnhanced,visualize,discover,discoverEnhanced,savedObjectsManagement,spaces,security,savedObjectsTagging,maps,lens,reporting,lists,encryptedSavedObjects,dashboardMode,cloud,upgradeAssistant,snapshotRestore,fleet,indexManagement,rollup,remoteClusters,crossClusterReplication,indexLifecycleManagement,enterpriseSearch,ml,beatsManagement,wazuh,transform,ingestPipelines,eventLog,actions,alerts,triggersActionsUi,stackAlerts,securitySolution,case,infra,monitoring,logstash,apm,uptime]"}
{"type":"log","@timestamp":"2021-07-15T00:21:18+00:00","tags":["info","plugins","taskManager"],"pid":1845073,"message":"TaskManager is identified by the Kibana UUID: d288334a-6d1f-4f72-ae5d-145112a02759"}
{"type":"log","@timestamp":"2021-07-15T00:21:18+00:00","tags":["warning","plugins","security","config"],"pid":1845073,"message":"Session cookies will be transmitted over insecure connections. This is not recommended."}
{"type":"log","@timestamp":"2021-07-15T00:21:18+00:00","tags":["warning","plugins","reporting","config"],"pid":1845073,"message":"Chromium sandbox provides an additional layer of protection, but is not supported for Linux CentOS 7.9.2009 OS. Automatically setting 'xpack.reporting.capture.browser.chromium.disableSandbox: true'."}
{"type":"log","@timestamp":"2021-07-15T00:21:18+00:00","tags":["info","plugins","monitoring","monitoring"],"pid":1845073,"message":"config sourced from: production cluster"}
{"type":"log","@timestamp":"2021-07-15T00:21:18+00:00","tags":["info","savedobjects-service"],"pid":1845073,"message":"Waiting until all Elasticsearch nodes are compatible with Kibana before starting saved objects migrations..."}
{"type":"log","@timestamp":"2021-07-15T00:21:19+00:00","tags":["info","savedobjects-service"],"pid":1845073,"message":"Starting saved objects migrations"}
{"type":"log","@timestamp":"2021-07-15T00:21:19+00:00","tags":["info","savedobjects-service"],"pid":1845073,"message":"Creating index .kibana_task_manager_1."}
{"type":"log","@timestamp":"2021-07-15T00:21:19+00:00","tags":["warning","savedobjects-service"],"pid":1845073,"message":"Unable to connect to Elasticsearch. Error: validation_exception"}
{"type":"log","@timestamp":"2021-07-15T00:21:19+00:00","tags":["fatal","root"],"pid":1845073,"message":"ResponseError: validation_exception\n at onBody (/usr/share/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:333:23)\n at IncomingMessage.onEnd (/usr/share/kibana/node_modules/@elastic/elasticsearch/lib/Transport.js:260:11)\n at IncomingMessage.emit (events.js:327:22)\n at endReadableNT (internal/streams/readable.js:1327:12)\n at processTicksAndRejections (internal/process/task_queues.js:80:21) {\n meta: {\n body: { error: [Object], status: 400 },\n statusCode: 400,\n headers: {\n 'content-type': 'application/json; charset=UTF-8',\n 'content-length': '379'\n },\n meta: {\n context: null,\n request: [Object],\n name: 'elasticsearch-js',\n connection: [Object],\n attempts: 0,\n aborted: false\n }\n }\n}"}
{"type":"log","@timestamp":"2021-07-15T00:21:19+00:00","tags":["info","plugins-system"],"pid":1845073,"message":"Stopping all plugins."}
{"type":"log","@timestamp":"2021-07-15T00:21:19+00:00","tags":["info","plugins","monitoring","monitoring","kibana-monitoring"],"pid":1845073,"message":"Monitoring stats collection is stopped"}
{"type":"log","@timestamp":"2021-07-15T00:21:19+00:00","tags":["info","savedobjects-service"],"pid":1845073,"message":"Creating index .kibana_1."}
{"type":"log","@timestamp":"2021-07-15T00:21:19+00:00","tags":["warning","savedobjects-service"],"pid":1845073,"message":"Unable to connect to Elasticsearch. Error: validation_exception"}
{"type":"log","@timestamp":"2021-07-15T00:21:48+00:00","tags":["warning","plugins","licensing"],"pid":1845073,"message":"License information could not be obtained from Elasticsearch due to Error: Cluster client cannot be used after it has been closed. error"}
{"type":"log","@timestamp":"2021-07-15T00:22:18+00:00","tags":["warning","plugins","licensing"],"pid":1845073,"message":"License information could not be obtained from Elasticsearch due to Error: Cluster client cannot be used after it has been closed. error"}
{"type":"log","@timestamp":"2021-07-15T00:22:48+00:00","tags":["warning","plugins","licensing"],"pid":1845073,"message":"License information could not be obtained from Elasticsearch due to Error: Cluster client cannot be u

```

---

<div class="post-metadata">

**Author:** ![dbernard](https://avatars.discourse-cdn.com/v4/letter/d/d9b06d/32.png) [@dbernard](https://discuss.elastic.co/u/dbernard)\
**Post date:** [July 15, 2021, 12:13am UTC](https://discuss.elastic.co/t/kibana-cannot-perform-migrations-due-to-validation-error/278678/4 "2021-07-15T00:13:50Z")

</div>

kibana.yml is minimal

```auto
server.port: 5601
server.name: "elk101"
elasticsearch.hosts: ["http://localhost:9200"]
elasticsearch.requestTimeout: 3000000 

server.host: "0.0.0.0"

xpack.security.enabled: true
elasticsearch.username: elastic
elasticsearch.password: changeme

uiSettings.overrides.defaultRoute: "/app/wazuh"
elasticsearch.ssl.verificationMode: none

```

---

<div class="post-metadata">

**Author:** ![dbernard](https://avatars.discourse-cdn.com/v4/letter/d/d9b06d/32.png) [@dbernard](https://discuss.elastic.co/u/dbernard)\
**Post date:** [July 15, 2021, 3:13pm UTC](https://discuss.elastic.co/t/kibana-cannot-perform-migrations-due-to-validation-error/278678/5 "2021-07-15T15:13:26Z")

</div>

Looks like the problem was with the upgraded elasticsearch not having a large enough maximum shards per node.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 12, 2021, 3:14pm UTC](https://discuss.elastic.co/t/kibana-cannot-perform-migrations-due-to-validation-error/278678/6 "2021-08-12T15:14:24Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.