Hi community,
for testing purposes I copied a VM with Ubuntu 18.04.6.
On that machine everything worked fine.
I upgraded the Ubuntu to 21.10 (the underlying Kibana and Elasticsearch had been updated too).
Now I have an issue accessing Kibana.
When accessing my kibana instance I am getting
'Kibana server is not ready yet'
So I checked the kibana logs.
{"type":"log","@timestamp":"2022-03-25T09:56:25+01:00","tags":["warning","config","deprecation"],"pid":11911,"message":"Starting in 8.0, the Kibana logging format will be changing. This may affect you if you are doing any special handling of your Kibana logs, such as ingesting logs into Elasticsearch for further analysis. If you are using the new logging configuration, you are already receiving logs in both old and new formats, and the old format will simply be going away. If you are not yet using the new logging configuration, the log format will change upon upgrade to 8.0. Beginning in 8.0, the format of JSON logs will be ECS-compatible JSON, and the default pattern log format will be configurable with our new logging system. Please refer to the documentation for more information about the new logging format."}
{"type":"log","@timestamp":"2022-03-25T09:56:25+01:00","tags":["warning","config","deprecation"],"pid":11911,"message":"The default mechanism for Reporting privileges will work differently in future versions, which will affect the behavior of this cluster. Set \"xpack.reporting.roles.enabled\" to \"false\" to adopt the future behavior before upgrading."}
{"type":"log","@timestamp":"2022-03-25T09:56:25+01:00","tags":["warning","config","deprecation"],"pid":11911,"message":"User sessions will automatically time out after 8 hours of inactivity starting in 8.0. Override this value to change the timeout."}
{"type":"log","@timestamp":"2022-03-25T09:56:25+01:00","tags":["warning","config","deprecation"],"pid":11911,"message":"Users are automatically required to log in again after 30 days starting in 8.0. Override this value to change the timeout."}
{"type":"log","@timestamp":"2022-03-25T09:56:25+01:00","tags":["info","plugins-system","standard"],"pid":11911,"message":"Setting up [113] plugins: [translations,licensing,globalSearch,globalSearchProviders,features,licenseApiGuard,code,usageCollection,xpackLegacy,taskManager,telemetryCollectionManager,telemetryCollectionXpack,kibanaUsageCollection,share,embeddable,uiActionsEnhanced,screenshotMode,banners,telemetry,newsfeed,mapsEms,mapsLegacy,kibanaLegacy,fieldFormats,expressions,dataViews,charts,esUiShared,bfetch,data,savedObjects,presentationUtil,expressionShape,expressionRevealImage,expressionRepeatImage,expressionMetric,expressionImage,customIntegrations,home,searchprofiler,painlessLab,grokdebugger,management,watcher,licenseManagement,advancedSettings,spaces,security,savedObjectsTagging,reporting,canvas,lists,ingestPipelines,fileUpload,encryptedSavedObjects,dataEnhanced,cloud,snapshotRestore,eventLog,actions,alerting,triggersActionsUi,transform,stackAlerts,ruleRegistry,visualizations,visTypeXy,visTypeVislib,visTypeVega,visTypeTimelion,visTypeTagcloud,visTypeTable,visTypePie,visTypeMetric,visTypeMarkdown,tileMap,regionMap,expressionTagcloud,expressionMetricVis,console,graph,fleet,indexManagement,remoteClusters,crossClusterReplication,indexLifecycleManagement,dashboard,maps,dashboardMode,dashboardEnhanced,visualize,visTypeTimeseries,rollup,indexPatternFieldEditor,lens,cases,timelines,discover,osquery,observability,discoverEnhanced,dataVisualizer,ml,uptime,securitySolution,infra,upgradeAssistant,monitoring,logstash,enterpriseSearch,apm,savedObjectsManagement,indexPatternManagement]"}
{"type":"log","@timestamp":"2022-03-25T09:56:25+01:00","tags":["info","plugins","taskManager"],"pid":11911,"message":"TaskManager is identified by the Kibana UUID: 4ab39e6e-dac1-475e-b903-9514aa503907"}
{"type":"log","@timestamp":"2022-03-25T09:56:26+01:00","tags":["warning","plugins","security","config"],"pid":11911,"message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command."}
{"type":"log","@timestamp":"2022-03-25T09:56:26+01:00","tags":["warning","plugins","security","config"],"pid":11911,"message":"Session cookies will be transmitted over insecure connections. This is not recommended."}
{"type":"log","@timestamp":"2022-03-25T09:56:26+01:00","tags":["warning","plugins","security","config"],"pid":11911,"message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command."}
{"type":"log","@timestamp":"2022-03-25T09:56:26+01:00","tags":["warning","plugins","security","config"],"pid":11911,"message":"Session cookies will be transmitted over insecure connections. This is not recommended."}
{"type":"log","@timestamp":"2022-03-25T09:56:26+01:00","tags":["warning","plugins","reporting","config"],"pid":11911,"message":"Generating a random key for xpack.reporting.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.reporting.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command."}
{"type":"log","@timestamp":"2022-03-25T09:56:26+01:00","tags":["warning","plugins","encryptedSavedObjects"],"pid":11911,"message":"Saved objects encryption key is not set. This will severely limit Kibana functionality. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command."}
{"type":"log","@timestamp":"2022-03-25T09:56:26+01:00","tags":["warning","plugins","actions"],"pid":11911,"message":"APIs are disabled because the Encrypted Saved Objects plugin is missing encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command."}
{"type":"log","@timestamp":"2022-03-25T09:56:26+01:00","tags":["warning","plugins","alerting"],"pid":11911,"message":"APIs are disabled because the Encrypted Saved Objects plugin is missing encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command."}
{"type":"log","@timestamp":"2022-03-25T09:56:26+01:00","tags":["info","plugins","ruleRegistry"],"pid":11911,"message":"Installing common resources shared between all indices"}
{"type":"log","@timestamp":"2022-03-25T09:56:26+01:00","tags":["info","plugins","reporting","config"],"pid":11911,"message":"Chromium sandbox provides an additional layer of protection, and is supported for Linux Ubuntu 21.10 OS. Automatically enabling Chromium sandbox."}
{"type":"log","@timestamp":"2022-03-25T09:56:28+01:00","tags":["info","savedobjects-service"],"pid":11911,"message":"Waiting until all Elasticsearch nodes are compatible with Kibana before starting saved objects migrations..."}
{"type":"log","@timestamp":"2022-03-25T09:56:28+01:00","tags":["info","savedobjects-service"],"pid":11911,"message":"Starting saved objects migrations"}
{"type":"log","@timestamp":"2022-03-25T09:56:28+01:00","tags":["info","savedobjects-service"],"pid":11911,"message":"[.kibana] INIT -> WAIT_FOR_YELLOW_SOURCE. took: 203ms."}
{"type":"log","@timestamp":"2022-03-25T09:56:28+01:00","tags":["info","savedobjects-service"],"pid":11911,"message":"[.kibana_task_manager] INIT -> WAIT_FOR_YELLOW_SOURCE. took: 204ms."}
{"type":"log","@timestamp":"2022-03-25T09:56:28+01:00","tags":["info","savedobjects-service"],"pid":11911,"message":"[.kibana] WAIT_FOR_YELLOW_SOURCE -> CHECK_UNKNOWN_DOCUMENTS. took: 103ms."}
{"type":"log","@timestamp":"2022-03-25T09:56:29+01:00","tags":["info","savedobjects-service"],"pid":11911,"message":"[.kibana] CHECK_UNKNOWN_DOCUMENTS -> SET_SOURCE_WRITE_BLOCK. took: 64ms."}
{"type":"log","@timestamp":"2022-03-25T09:56:29+01:00","tags":["info","savedobjects-service"],"pid":11911,"message":"[.kibana] SET_SOURCE_WRITE_BLOCK -> CALCULATE_EXCLUDE_FILTERS. took: 15ms."}
{"type":"log","@timestamp":"2022-03-25T09:56:29+01:00","tags":["error","savedobjects-service"],"pid":11911,"message":"[.kibana] Action failed with 'search_phase_execution_exception: [no_shard_available_action_exception] Reason: null'. Retrying attempt 1 in 2 seconds."}
{"type":"log","@timestamp":"2022-03-25T09:56:29+01:00","tags":["info","savedobjects-service"],"pid":11911,"message":"[.kibana] CALCULATE_EXCLUDE_FILTERS -> CALCULATE_EXCLUDE_FILTERS. took: 33ms."}
{"type":"log","@timestamp":"2022-03-25T09:56:31+01:00","tags":["error","savedobjects-service"],"pid":11911,"message":"[.kibana] Action failed with 'search_phase_execution_exception: [no_shard_available_action_exception] Reason: null'. Retrying attempt 2 in 4 seconds."}
{"type":"log","@timestamp":"2022-03-25T09:56:31+01:00","tags":["info","savedobjects-service"],"pid":11911,"message":"[.kibana] CALCULATE_EXCLUDE_FILTERS -> CALCULATE_EXCLUDE_FILTERS. took: 2049ms."}
{"type":"log","@timestamp":"2022-03-25T09:56:35+01:00","tags":["error","savedobjects-service"],"pid":11911,"message":"[.kibana] Action failed with 'search_phase_execution_exception: [no_shard_available_action_exception] Reason: null'. Retrying attempt 3 in 8 seconds."}
{"type":"log","@timestamp":"2022-03-25T09:56:35+01:00","tags":["info","savedobjects-service"],"pid":11911,"message":"[.kibana] CALCULATE_EXCLUDE_FILTERS -> CALCULATE_EXCLUDE_FILTERS. took: 4017ms."}
{"type":"log","@timestamp":"2022-03-25T09:56:43+01:00","tags":["error","savedobjects-service"],"pid":11911,"message":"[.kibana] Action failed with 'search_phase_execution_exception: [no_shard_available_action_exception] Reason: null'. Retrying attempt 4 in 16 seconds."}
{"type":"log","@timestamp":"2022-03-25T09:56:43+01:00","tags":["info","savedobjects-service"],"pid":11911,"message":"[.kibana] CALCULATE_EXCLUDE_FILTERS -> CALCULATE_EXCLUDE_FILTERS. took: 8019ms."}
{"type":"log","@timestamp":"2022-03-25T09:56:59+01:00","tags":["error","savedobjects-service"],"pid":11911,"message":"[.kibana] Action failed with 'search_phase_execution_exception: [no_shard_available_action_exception] Reason: null'. Retrying attempt 5 in 32 seconds."}
{"type":"log","@timestamp":"2022-03-25T09:56:59+01:00","tags":["info","savedobjects-service"],"pid":11911,"message":"[.kibana] CALCULATE_EXCLUDE_FILTERS -> CALCULATE_EXCLUDE_FILTERS. took: 16033ms."}
{"type":"log","@timestamp":"2022-03-25T09:57:31+01:00","tags":["error","savedobjects-service"],"pid":11911,"message":"[.kibana] Action failed with 'search_phase_execution_exception: [no_shard_available_action_exception] Reason: null'. Retrying attempt 6 in 64 seconds."}
{"type":"log","@timestamp":"2022-03-25T09:57:31+01:00","tags":["info","savedobjects-service"],"pid":11911,"message":"[.kibana] CALCULATE_EXCLUDE_FILTERS -> CALCULATE_EXCLUDE_FILTERS. took: 32040ms."}
What can I check to get Kibana server ready?
thank you
