Kibana not starting. [.kibana_task_manager] Action failed with 'Request timed out'

rkimera · November 1, 2022, 1:59pm

I am able to run the elasticsearch server at port 9200. But this kibana server is failing to start throwing an error. I have downgraded to version 7. x.x from 8. x.x but still get the same error.

I am using a Mac os.

</ log   [16:51:27.493] [info][plugins-service] Plugin "metricsEntities" is disabled.
  log   [16:51:27.777] [info][server][Preboot][http] http server running at http://localhost:5601
  log   [16:51:27.885] [warning][config][deprecation] Starting in 8.0, the Kibana logging format will be changing. This may affect you if you are doing any special handling of your Kibana logs, such as ingesting logs into Elasticsearch for further analysis. If you are using the new logging configuration, you are already receiving logs in both old and new formats, and the old format will simply be going away. If you are not yet using the new logging configuration, the log format will change upon upgrade to 8.0. Beginning in 8.0, the format of JSON logs will be ECS-compatible JSON, and the default pattern log format will be configurable with our new logging system. Please refer to the documentation for more information about the new logging format.
  log   [16:51:27.888] [warning][config][deprecation] The default mechanism for Reporting privileges will work differently in future versions, which will affect the behavior of this cluster. Set "xpack.reporting.roles.enabled" to "false" to adopt the future behavior before upgrading.
  log   [16:51:27.895] [warning][config][deprecation] User sessions will automatically time out after 8 hours of inactivity starting in 8.0. Override this value to change the timeout.
  log   [16:51:27.902] [warning][config][deprecation] Users are automatically required to log in again after 30 days starting in 8.0. Override this value to change the timeout.
  log   [16:51:28.267] [info][plugins-system][standard] Setting up [113] plugins: [translations,licensing,globalSearch,globalSearchProviders,features,licenseApiGuard,code,usageCollection,xpackLegacy,taskManager,telemetryCollectionManager,telemetryCollectionXpack,kibanaUsageCollection,share,embeddable,uiActionsEnhanced,screenshotMode,banners,telemetry,newsfeed,mapsEms,mapsLegacy,kibanaLegacy,fieldFormats,expressions,dataViews,charts,esUiShared,bfetch,data,savedObjects,presentationUtil,expressionShape,expressionRevealImage,expressionRepeatImage,expressionMetric,expressionImage,customIntegrations,home,searchprofiler,painlessLab,grokdebugger,management,watcher,licenseManagement,advancedSettings,spaces,security,savedObjectsTagging,reporting,canvas,lists,ingestPipelines,fileUpload,encryptedSavedObjects,dataEnhanced,cloud,snapshotRestore,eventLog,actions,alerting,triggersActionsUi,transform,stackAlerts,ruleRegistry,visualizations,visTypeXy,visTypeVislib,visTypeVega,visTypeTimelion,visTypeTagcloud,visTypeTable,visTypePie,visTypeMetric,visTypeMarkdown,tileMap,regionMap,expressionTagcloud,expressionMetricVis,console,graph,fleet,indexManagement,remoteClusters,crossClusterReplication,indexLifecycleManagement,dashboard,maps,dashboardMode,dashboardEnhanced,visualize,visTypeTimeseries,rollup,indexPatternFieldEditor,lens,cases,timelines,discover,osquery,observability,discoverEnhanced,dataVisualizer,ml,uptime,securitySolution,infra,upgradeAssistant,monitoring,logstash,enterpriseSearch,apm,savedObjectsManagement,indexPatternManagement]
  log   [16:51:28.309] [info][plugins][taskManager] TaskManager is identified by the Kibana UUID: 72b2a71b-7601-4fb3-9f03-e2b8239fd629
  log   [16:51:28.622] [warning][config][plugins][security] Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
  log   [16:51:28.623] [warning][config][plugins][security] Session cookies will be transmitted over insecure connections. This is not recommended.
  log   [16:51:28.715] [warning][config][plugins][security] Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
  log   [16:51:28.715] [warning][config][plugins][security] Session cookies will be transmitted over insecure connections. This is not recommended.
  log   [16:51:28.796] [warning][config][plugins][reporting] Generating a random key for xpack.reporting.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.reporting.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
  log   [16:51:28.817] [warning][encryptedSavedObjects][plugins] Saved objects encryption key is not set. This will severely limit Kibana functionality. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
  log   [16:51:28.866] [warning][actions][plugins] APIs are disabled because the Encrypted Saved Objects plugin is missing encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
  log   [16:51:28.992] [warning][alerting][plugins] APIs are disabled because the Encrypted Saved Objects plugin is missing encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
  log   [16:51:29.017] [info][plugins][ruleRegistry] Installing common resources shared between all indices
  log   [16:51:30.066] [info][config][plugins][reporting] Chromium sandbox provides an additional layer of protection, and is supported for Darwin OS. Automatically enabling Chromium sandbox.
  log   [16:51:30.542] [info][savedobjects-service] Waiting until all Elasticsearch nodes are compatible with Kibana before starting saved objects migrations...
  log   [16:51:30.563] [info][savedobjects-service] Starting saved objects migrations
  log   [16:51:30.728] [info][savedobjects-service] [.kibana] INIT -> CREATE_NEW_TARGET. took: 73ms.
  log   [16:51:30.736] [info][savedobjects-service] [.kibana_task_manager] INIT -> CREATE_NEW_TARGET. took: 64ms.
  log   [16:53:30.825] [error][savedobjects-service] [.kibana_task_manager] Action failed with 'Request timed out'. Retrying attempt 1 in 2 seconds.
  log   [16:53:30.833] [info][savedobjects-service] [.kibana_task_manager] CREATE_NEW_TARGET -> CREATE_NEW_TARGET. took: 120089ms.
  log   [16:53:30.836] [error][savedobjects-service] [.kibana] Action failed with 'Request timed out'. Retrying attempt 1 in 2 seconds.
  log   [16:53:30.836] [info][savedobjects-service] [.kibana] CREATE_NEW_TARGET -> CREATE_NEW_TARGET. took: 120108ms.
$  log   [16:55:32.865] [error][savedobjects-service] [.kibana_task_manager] Action failed with 'Request timed out'. Retrying attempt 2 in 4 seconds.
  log   [16:55:32.870] [info][savedobjects-service] [.kibana_task_manager] CREATE_NEW_TARGET -> CREATE_NEW_TARGET. took: 122040ms.
  log   [16:55:32.872] [error][savedobjects-service] [.kibana] Action failed with 'Request timed out'. Retrying attempt 2 in 4 seconds.
  log   [16:55:32.872] [info][savedobjects-service] [.kibana] CREATE_NEW_TARGET -> CREATE_NEW_TARGET. took: 122036ms.
  log   [16:57:36.895] [error][savedobjects-service] [.kibana_task_manager] Action failed with 'Request timed out'. Retrying attempt 3 in 8 seconds.
  log   [16:57:36.898] [info][savedobjects-service] [.kibana_task_manager] CREATE_NEW_TARGET -> CREATE_NEW_TARGET. took: 124030ms.
  log   [16:57:36.906] [error][savedobjects-service] [.kibana] Action failed with 'Request timed out'. Retrying attempt 3 in 8 seconds.
  log   [16:57:36.906] [info][savedobjects-service] [.kibana] CREATE_NEW_TARGET -> CREATE_NEW_TARGET. took: 124033ms.>

rkimera · November 1, 2022, 2:01pm

i have tried looking at the different suggestion posted, but not able to get a specific solution

Julien · November 1, 2022, 5:11pm

Can you check if the index was created and is at least yellow health

GET _cat/indices/.kibana_task_manager*

If the index is in red, check why with allocation explain and fix any issue (potentially you reached low watermark on all eligible nodes to give just one example)

If the index is currently not red or is not created, potentially restart kibana and check the logs of the elected master for elasticssearch cluster at same timestamp where you see the error creating kibana task manager system index

rkimera · November 1, 2022, 9:00pm

how do I run this command? since I have no access to the dev tool, given that kibana isn't running.
I am new to Elasticsearch! beginner level

Julien · November 2, 2022, 9:05am

Click on copy as curl from the documentation example and add -u <username> to the command (if elasticsearch is secured), adapt the URL to your Elasticsearch node. You can use any other rest client tool

rkimera · November 5, 2022, 11:06am

the index was not created.
And the error in the terminal when running bin/kibana is
</
[.kibana] CREATE_NEW_TARGET -> CREATE_NEW_TARGET. took: 124029ms.log [14:03:43.627] [error][savedobjects-service] [.kibana_task_manager] Action failed with 'Request timed out'. Retrying attempt 4 in 16 seconds. >

rkimera · November 5, 2022, 11:23am

On running the command, not red, or yellow :

curl -X GET "localhost:9200/_cat/indices/my-index-*?v=true&s=index&pretty"

rkimera · November 5, 2022, 11:31am

@Julien Elasticsearch instance is running fine! I have been able to create an index. The issue is on connecting the kibana instance(running bin/kibana). That is when I get the error!

system · January 27, 2023, 7:24pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.