Kibana is not getting up

Hello All,

I am trying to connect my elasticsearch hosted in aws cloud with kibana(on premise host).
My elastic comes up properly but kibana is not getting up.

always same message:
image

Logs:

{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:16:59.213+02:00","message":"Stopping all plugins.","log":{"level":"INFO","logger":"plugins-system.preboot"},"process":{"pid":20479}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:05.599+02:00","message":"Kibana process configured with roles: [background_tasks, ui]","log":{"level":"INFO","logger":"node"},"process":{"pid":28733},"trace":{"id":"9855a83a6baa8e70e0e4a8c56bde2b94"},"transaction":{"id":"775ae0c961b4bce7"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:40.127+02:00","message":"Plugin \"cloudChat\" is disabled.","log":{"level":"INFO","logger":"plugins-service"},"process":{"pid":28733},"trace":{"id":"9855a83a6baa8e70e0e4a8c56bde2b94"},"transaction":{"id":"775ae0c961b4bce7"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:40.127+02:00","message":"Plugin \"cloudExperiments\" is disabled.","log":{"level":"INFO","logger":"plugins-service"},"process":{"pid":28733},"trace":{"id":"9855a83a6baa8e70e0e4a8c56bde2b94"},"transaction":{"id":"775ae0c961b4bce7"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:40.127+02:00","message":"Plugin \"cloudFullStory\" is disabled.","log":{"level":"INFO","logger":"plugins-service"},"process":{"pid":28733},"trace":{"id":"9855a83a6baa8e70e0e4a8c56bde2b94"},"transaction":{"id":"775ae0c961b4bce7"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:40.127+02:00","message":"Plugin \"cloudGainsight\" is disabled.","log":{"level":"INFO","logger":"plugins-service"},"process":{"pid":28733},"trace":{"id":"9855a83a6baa8e70e0e4a8c56bde2b94"},"transaction":{"id":"775ae0c961b4bce7"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:40.139+02:00","message":"Plugin \"profiling\" is disabled.","log":{"level":"INFO","logger":"plugins-service"},"process":{"pid":28733},"trace":{"id":"9855a83a6baa8e70e0e4a8c56bde2b94"},"transaction":{"id":"775ae0c961b4bce7"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:40.285+02:00","message":"http server running at http://wvcnctd02ds001.dev.kpdm01.group.wvg:5601","log":{"level":"INFO","logger":"http.server.Preboot"},"process":{"pid":28733},"trace":{"id":"9855a83a6baa8e70e0e4a8c56bde2b94"},"transaction":{"id":"775ae0c961b4bce7"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:40.411+02:00","message":"Setting up [1] plugins: [interactiveSetup]","log":{"level":"INFO","logger":"plugins-system.preboot"},"process":{"pid":28733},"trace":{"id":"9855a83a6baa8e70e0e4a8c56bde2b94"},"transaction":{"id":"775ae0c961b4bce7"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:40.498+02:00","message":"Setting \"server.maxPayloadBytes\" has been replaced by \"server.maxPayload\"","log":{"level":"WARN","logger":"config.deprecation"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:40.499+02:00","message":"The default mechanism for Reporting privileges will work differently in future versions, which will affect the behavior of this cluster. Set \"xpack.reporting.roles.enabled\" to \"false\" to adopt the future behavior before upgrading.","log":{"level":"WARN","logger":"config.deprecation"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:41.046+02:00","message":"Setting up [132] plugins: [translations,monitoringCollection,licensing,globalSearch,globalSearchProviders,features,mapsEms,licenseApiGuard,customBranding,usageCollection,taskManager,cloud,guidedOnboarding,telemetryCollectionManager,telemetryCollectionXpack,kibanaUsageCollection,share,screenshotMode,banners,newsfeed,ftrApis,fieldFormats,expressions,screenshotting,dataViews,charts,esUiShared,customIntegrations,home,searchprofiler,painlessLab,grokdebugger,management,cloudDataMigration,advancedSettings,spaces,security,snapshotRestore,lists,encryptedSavedObjects,telemetry,licenseManagement,files,eventLog,actions,notifications,console,contentManagement,bfetch,data,watcher,fileUpload,ingestPipelines,ecsDataQualityDashboard,alerting,unifiedSearch,unifiedFieldList,savedSearch,savedObjects,graph,savedObjectsTagging,savedObjectsManagement,eventAnnotation,embeddable,reporting,uiActionsEnhanced,presentationUtil,expressionShape,expressionRevealImage,expressionRepeatImage,expressionMetric,expressionImage,controls,dataViewFieldEditor,triggersActionsUi,transform,stackConnectors,stackAlerts,ruleRegistry,visualizations,canvas,visTypeXy,visTypeVislib,visTypeVega,visTypeTimeseries,visTypeTimelion,visTypeTagcloud,visTypeTable,visTypeMetric,visTypeHeatmap,visTypeMarkdown,dashboard,dashboardEnhanced,expressionXY,expressionTagcloud,expressionPartitionVis,visTypePie,expressionMetricVis,expressionLegacyMetricVis,expressionHeatmap,expressionGauge,lens,maps,cases,timelines,sessionView,kubernetesSecurity,threatIntelligence,aiops,discover,observability,fleet,osquery,indexManagement,rollup,remoteClusters,crossClusterReplication,indexLifecycleManagement,cloudSecurityPosture,discoverEnhanced,dataVisualizer,ml,synthetics,securitySolution,infra,upgradeAssistant,monitoring,logstash,enterpriseSearch,apm,visTypeGauge,dataViewManagement]","log":{"level":"INFO","logger":"plugins-system.standard"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:41.080+02:00","message":"CustomBrandingService registering plugin: customBranding","log":{"level":"INFO","logger":"custom-branding-service"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:41.089+02:00","message":"TaskManager is identified by the Kibana UUID: b621a760-aefe-4718-a86e-2271ad995a67","log":{"level":"INFO","logger":"plugins.taskManager"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:41.278+02:00","message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.","log":{"level":"WARN","logger":"plugins.security.config"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:41.279+02:00","message":"Session cookies will be transmitted over insecure connections. This is not recommended.","log":{"level":"WARN","logger":"plugins.security.config"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:41.339+02:00","message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.","log":{"level":"WARN","logger":"plugins.security.config"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:41.339+02:00","message":"Session cookies will be transmitted over insecure connections. This is not recommended.","log":{"level":"WARN","logger":"plugins.security.config"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:41.362+02:00","message":"Saved objects encryption key is not set. This will severely limit Kibana functionality. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.","log":{"level":"WARN","logger":"plugins.encryptedSavedObjects"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:41.387+02:00","message":"APIs are disabled because the Encrypted Saved Objects plugin is missing encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.","log":{"level":"WARN","logger":"plugins.actions"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:41.400+02:00","message":"Email Service Error: Email connector not specified.","log":{"level":"INFO","logger":"plugins.notifications"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:41.593+02:00","message":"APIs are disabled because the Encrypted Saved Objects plugin is missing encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.","log":{"level":"WARN","logger":"plugins.alerting"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:41.672+02:00","message":"Generating a random key for xpack.reporting.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.reporting.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.","log":{"level":"WARN","logger":"plugins.reporting.config"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:41.725+02:00","message":"Installing common resources shared between all indices","log":{"level":"INFO","logger":"plugins.ruleRegistry"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:42.378+02:00","message":"Registered task successfully [Task: cloud_security_posture-stats_task]","log":{"level":"INFO","logger":"plugins.cloudSecurityPosture"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:43.211+02:00","message":"Chromium sandbox provides an additional layer of protection, but is not supported for Linux Red Hat Linux 7.6 OS. Automatically setting 'xpack.screenshotting.browser.chromium.disableSandbox: true'.","log":{"level":"WARN","logger":"plugins.screenshotting.config"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:43.507+02:00","message":"Unable to retrieve version information from Elasticsearch nodes. security_exception\n\tRoot causes:\n\t\tsecurity_exception: missing authentication credentials for REST request [/_nodes?filter_path=nodes.*.version%2Cnodes.*.http.publish_address%2Cnodes.*.ip]","log":{"level":"ERROR","logger":"elasticsearch-service"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.0"},"@timestamp":"2023-05-25T14:17:44.617+02:00","message":"Browser executable: /l/app/mis/tmp/kibana-8.7.1/x-pack/plugins/screenshotting/chromium/headless_shell-linux_x64/headless_shell","log":{"level":"INFO","logger":"plugins.screenshotting.chromium"},"process":{"pid":28733},"trace":{"id":"40004c2a8cbbc3fc909ff832e59eccaf"},"transaction":{"id":"5f3088dc75244a87"}}

kibana.yml:

server.maxPayloadBytes: 2097152
server.name: mis
elasticsearch.hosts:
- https://abc:443
elasticsearch.ssl.certificateAuthorities: /l/ppp/mis/tmp/kibana-8.7.1/ece_proxy_root_ca.pem
server.port: 5601
server.host: abc
logging.appenders.default:
  type: file
  fileName: /l/ppp/mis/tmp/kibana.log
  layout:
    type: json

elastic/kibana: 8.7.1 version

Looks like you are missing the elasticsearch credentials in your kibana.yml

Hello @stephenb ,

I've used username and password also earlier,didn't worked.

Thanks

Perhaps that was not correct either... Okay have you checked connectivity do the curl from the Kibana box to elasticsearch box

If you are using elastic 8.7 you cannot use the elastic user you need use the kibana_system user

curl -k -v -u:kibana_system:password https://elastic:port

If that works then try with the --cacert option to check the cert

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.