Kibana is not getting up

PRASHANT_MEHTA · May 25, 2023, 12:37pm

Hello All,

I am trying to connect my elasticsearch hosted in aws cloud with kibana(on premise host).
My elastic comes up properly but kibana is not getting up.

always same message:

Logs:

{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:16:59.213+02:00","message":"Stopping all plugins.","log":{"level":"INFO","logger":"plugins-system.preboot"},"process":{"pid":20479}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:05.599+02:00","message":"Kibana process configured with roles: [background_tasks, ui]","log":{"level":"INFO","logger":"node"},"process":{"pid":28733},"trace":{"id":"9855a83a6baa8e70e0e4a8c56bde2b94"},"transaction":{"id":"775ae0c961b4bce7"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:40.127+02:00","message":"Plugin \"cloudChat\" is disabled.","log":{"level":"INFO","logger":"plugins-service"},"process":{"pid":28733},"trace":{"id":"9855a83a6baa8e70e0e4a8c56bde2b94"},"transaction":{"id":"775ae0c961b4bce7"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:40.127+02:00","message":"Plugin \"cloudExperiments\" is disabled.","log":{"level":"INFO","logger":"plugins-service"},"process":{"pid":28733},"trace":{"id":"9855a83a6baa8e70e0e4a8c56bde2b94"},"transaction":{"id":"775ae0c961b4bce7"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:40.127+02:00","message":"Plugin \"cloudFullStory\" is disabled.","log":{"level":"INFO","logger":"plugins-service"},"process":{"pid":28733},"trace":{"id":"9855a83a6baa8e70e0e4a8c56bde2b94"},"transaction":{"id":"775ae0c961b4bce7"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:40.127+02:00","message":"Plugin \"cloudGainsight\" is disabled.","log":{"level":"INFO","logger":"plugins-service"},"process":{"pid":28733},"trace":{"id":"9855a83a6baa8e70e0e4a8c56bde2b94"},"transaction":{"id":"775ae0c961b4bce7"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:40.139+02:00","message":"Plugin \"profiling\" is disabled.","log":{"level":"INFO","logger":"plugins-service"},"process":{"pid":28733},"trace":{"id":"9855a83a6baa8e70e0e4a8c56bde2b94"},"transaction":{"id":"775ae0c961b4bce7"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:40.285+02:00","message":"http server running at http://wvcnctd02ds001.dev.kpdm01.group.wvg:5601","log":{"level":"INFO","logger":"http.server.Preboot"},"process":{"pid":28733},"trace":{"id":"9855a83a6baa8e70e0e4a8c56bde2b94"},"transaction":{"id":"775ae0c961b4bce7"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:40.411+02:00","message":"Setting up [1] plugins: [interactiveSetup]","log":{"level":"INFO","logger":"plugins-system.preboot"},"process":{"pid":28733},"trace":{"id":"9855a83a6baa8e70e0e4a8c56bde2b94"},"transaction":{"id":"775ae0c961b4bce7"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:40.498+02:00","message":"Setting \"server.maxPayloadBytes\" has been replaced by \"server.maxPayload\"","log":{"level":"WARN","logger":"config.deprecation"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:40.499+02:00","message":"The default mechanism for Reporting privileges will work differently in future versions, which will affect the behavior of this cluster. Set \"xpack.reporting.roles.enabled\" to \"false\" to adopt the future behavior before upgrading.","log":{"level":"WARN","logger":"config.deprecation"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:41.046+02:00","message":"Setting up [132] plugins: [translations,monitoringCollection,licensing,globalSearch,globalSearchProviders,features,mapsEms,licenseApiGuard,customBranding,usageCollection,taskManager,cloud,guidedOnboarding,telemetryCollectionManager,telemetryCollectionXpack,kibanaUsageCollection,share,screenshotMode,banners,newsfeed,ftrApis,fieldFormats,expressions,screenshotting,dataViews,charts,esUiShared,customIntegrations,home,searchprofiler,painlessLab,grokdebugger,management,cloudDataMigration,advancedSettings,spaces,security,snapshotRestore,lists,encryptedSavedObjects,telemetry,licenseManagement,files,eventLog,actions,notifications,console,contentManagement,bfetch,data,watcher,fileUpload,ingestPipelines,ecsDataQualityDashboard,alerting,unifiedSearch,unifiedFieldList,savedSearch,savedObjects,graph,savedObjectsTagging,savedObjectsManagement,eventAnnotation,embeddable,reporting,uiActionsEnhanced,presentationUtil,expressionShape,expressionRevealImage,expressionRepeatImage,expressionMetric,expressionImage,controls,dataViewFieldEditor,triggersActionsUi,transform,stackConnectors,stackAlerts,ruleRegistry,visualizations,canvas,visTypeXy,visTypeVislib,visTypeVega,visTypeTimeseries,visTypeTimelion,visTypeTagcloud,visTypeTable,visTypeMetric,visTypeHeatmap,visTypeMarkdown,dashboard,dashboardEnhanced,expressionXY,expressionTagcloud,expressionPartitionVis,visTypePie,expressionMetricVis,expressionLegacyMetricVis,expressionHeatmap,expressionGauge,lens,maps,cases,timelines,sessionView,kubernetesSecurity,threatIntelligence,aiops,discover,observability,fleet,osquery,indexManagement,rollup,remoteClusters,crossClusterReplication,indexLifecycleManagement,cloudSecurityPosture,discoverEnhanced,dataVisualizer,ml,synthetics,securitySolution,infra,upgradeAssistant,monitoring,logstash,enterpriseSearch,apm,visTypeGauge,dataViewManagement]","log":{"level":"INFO","logger":"plugins-system.standard"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:41.080+02:00","message":"CustomBrandingService registering plugin: customBranding","log":{"level":"INFO","logger":"custom-branding-service"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:41.089+02:00","message":"TaskManager is identified by the Kibana UUID: b621a760-aefe-4718-a86e-2271ad995a67","log":{"level":"INFO","logger":"plugins.taskManager"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:41.278+02:00","message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.","log":{"level":"WARN","logger":"plugins.security.config"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:41.279+02:00","message":"Session cookies will be transmitted over insecure connections. This is not recommended.","log":{"level":"WARN","logger":"plugins.security.config"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:41.339+02:00","message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.","log":{"level":"WARN","logger":"plugins.security.config"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:41.339+02:00","message":"Session cookies will be transmitted over insecure connections. This is not recommended.","log":{"level":"WARN","logger":"plugins.security.config"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:41.362+02:00","message":"Saved objects encryption key is not set. This will severely limit Kibana functionality. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.","log":{"level":"WARN","logger":"plugins.encryptedSavedObjects"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:41.387+02:00","message":"APIs are disabled because the Encrypted Saved Objects plugin is missing encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.","log":{"level":"WARN","logger":"plugins.actions"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:41.400+02:00","message":"Email Service Error: Email connector not specified.","log":{"level":"INFO","logger":"plugins.notifications"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:41.593+02:00","message":"APIs are disabled because the Encrypted Saved Objects plugin is missing encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.","log":{"level":"WARN","logger":"plugins.alerting"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:41.672+02:00","message":"Generating a random key for xpack.reporting.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.reporting.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.","log":{"level":"WARN","logger":"plugins.reporting.config"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:41.725+02:00","message":"Installing common resources shared between all indices","log":{"level":"INFO","logger":"plugins.ruleRegistry"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:42.378+02:00","message":"Registered task successfully [Task: cloud_security_posture-stats_task]","log":{"level":"INFO","logger":"plugins.cloudSecurityPosture"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:43.211+02:00","message":"Chromium sandbox provides an additional layer of protection, but is not supported for Linux Red Hat Linux 7.6 OS. Automatically setting 'xpack.screenshotting.browser.chromium.disableSandbox: true'.","log":{"level":"WARN","logger":"plugins.screenshotting.config"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:43.507+02:00","message":"Unable to retrieve version information from Elasticsearch nodes. security_exception\n\tRoot causes:\n\t\tsecurity_exception: missing authentication credentials for REST request [/_nodes?filter_path=nodes.*.version%2Cnodes.*.http.publish_address%2Cnodes.*.ip]","log":{"level":"ERROR","logger":"elasticsearch-service"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:44.617+02:00","message":"Browser executable: /l/app/mis/tmp/kibana-8.7.1/x-pack/plugins/screenshotting/chromium/headless_shell-linux_x64/headless_shell","log":{"level":"INFO","logger":"plugins.screenshotting.chromium"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}

kibana.yml:

server.maxPayloadBytes: 2097152
server.name: mis
elasticsearch.hosts:
- https://abc:443
elasticsearch.ssl.certificateAuthorities: /l/ppp/mis/tmp/kibana-8.7.1/ece_proxy_root_ca.pem
server.port: 5601
server.host: abc
logging.appenders.default:
  type: file
  fileName: /l/ppp/mis/tmp/kibana.log
  layout:
    type: json

elastic/kibana: 8.7.1 version

stephenb · May 25, 2023, 1:54pm

Looks like you are missing the elasticsearch credentials in your kibana.yml

PRASHANT_MEHTA · May 26, 2023, 1:52pm

Hello @stephenb ,

I've used username and password also earlier,didn't worked.

Thanks

stephenb · May 26, 2023, 2:38pm

Perhaps that was not correct either... Okay have you checked connectivity do the curl from the Kibana box to elasticsearch box

If you are using elastic 8.7 you cannot use the elastic user you need use the kibana_system user

curl -k -v -u:kibana_system:password https://elastic:port

If that works then try with the --cacert option to check the cert

system · June 23, 2023, 2:38pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Not able to connect to ElasticSearch Cluster in AWS from Kibana Kibana	4	1837	March 9, 2020
Kibana server is not ready yet Kibana	10	4156	April 2, 2024
Kibana not communicating with Elasticsearch Kibana	5	2091	August 17, 2018
KIBANA UI is not coming up even all services are UP Without any error in Logs Kibana	6	284	March 22, 2024
Kibana is unable to connect to ElasticSearch Kibana	2	827	March 2, 2023

Kibana is not getting up

Related topics