Kiabana Server is not ready yet

Elastic Stack Kibana
1

Hi Team,

Can someone please help to translate this log below also advise with the fix.

Any help is greatly appreciated.

Thank you

"observability.threshold".","log":{"level":"INFO","logger":"plugins.alerting"},"process":{"pid":1056,"uptime":32.229934173},"trace":{"id":"98f61cc72d15b478213c6c0f199c4379"},"transaction":{"id":"7968df243d6cf760"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2024-06-19T14:50:22.091+01:00","message":"Registering resources for context "observability.uptime".","log":{"level":"INFO","logger":"plugins.alerting"},"process":{"pid":1056,"uptime":32.396241153},"trace":{"id":"98f61cc72d15b478213c6c0f199c4379"},"transaction":{"id":"7968df243d6cf760"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2024-06-19T14:50:22.492+01:00","message":"Skipping initialization of Profiling endpoints because 'profilingDataAccess' plugin is not available","log":{"level":"INFO","logger":"plugins.infra"},"process":{"pid":1056,"uptime":32.797415273},"trace":{"id":"98f61cc72d15b478213c6c0f199c4379"},"transaction":{"id":"7968df243d6cf760"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2024-06-19T14:50:22.493+01:00","message":"Registering resources for context "observability.logs".","log":{"level":"INFO","logger":"plugins.alerting"},"process":{"pid":1056,"uptime":32.798906833},"trace":{"id":"98f61cc72d15b478213c6c0f199c4379"},"transaction":{"id":"7968df243d6cf760"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2024-06-19T14:50:22.495+01:00","message":"Registering resources for context "observability.metrics".","log":{"level":"INFO","logger":"plugins.alerting"},"process":{"pid":1056,"uptime":32.800780128},"trace":{"id":"98f61cc72d15b478213c6c0f199c4379"},"transaction":{"id":"7968df243d6cf760"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2024-06-19T14:50:26.451+01:00","message":"Registering resources for context "security".","log":{"level":"INFO","logger":"plugins.alerting"},"process":{"pid":1056,"uptime":36.756913127},"trace":{"id":"98f61cc72d15b478213c6c0f199c4379"},"transaction":{"id":"7968df243d6cf760"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2024-06-19T14:50:26.514+01:00","message":"Server is NOT enabled","log":{"level":"INFO","logger":"plugins.assetManager"},"process":{"pid":1056,"uptime":36.819831936},"trace":{"id":"98f61cc72d15b478213c6c0f199c4379"},"transaction":{"id":"7968df243d6cf760"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2024-06-19T14:50:26.522+01:00","message":"Registering resources for context "observability.apm".","log":{"level":"INFO","logger":"plugins.alerting"},"process":{"pid":1056,"uptime":36.82726429},"trace":{"id":"98f61cc72d15b478213c6c0f199c4379"},"transaction":{"id":"7968df243d6cf760"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2024-06-19T14:50:27.036+01:00","message":"Unable to retrieve version information from Elasticsearch nodes. certificate has expired","log":{"level":"ERROR","logger":"elasticsearch-service"},"process":{"pid":1056,"uptime":37.341328283},"trace":{"id":"98f61cc72d15b478213c6c0f199c4379"},"transaction":{"id":"7968df243d6cf760"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.6.1"},"@timestamp":"2024-06-19T14:50:28.060+01:00","message":"Browser executable: /usr/share/kibana/node_modules/@kbn/screenshotting-plugin/chromium/headless_shell-linux_x64/headless_shell","log":{"level":"INFO","logger":"plugins.screenshotting.chromium"},"process":{"pid":1056,"uptime":38.365977472},"

Jun 19 14:50:21 smg-lsm-m01 kibana[1056]: [2024-06-19T14:50:21.924+01:00][INFO ][plugins.alerting] Registering resources for context "observability.threshold".
Jun 19 14:50:22 smg-lsm-m01 kibana[1056]: [2024-06-19T14:50:22.091+01:00][INFO ][plugins.alerting] Registering resources for context "observability.uptime".
Jun 19 14:50:22 smg-lsm-m01 kibana[1056]: [2024-06-19T14:50:22.492+01:00][INFO ][plugins.infra] Skipping initialization of Profiling endpoints because 'profilingDataAccess' plugin is not available
Jun 19 14:50:22 smg-lsm-m01 kibana[1056]: [2024-06-19T14:50:22.493+01:00][INFO ][plugins.alerting] Registering resources for context "observability.logs".
Jun 19 14:50:22 smg-lsm-m01 kibana[1056]: [2024-06-19T14:50:22.495+01:00][INFO ][plugins.alerting] Registering resources for context "observability.metrics".
Jun 19 14:50:26 smg-lsm-m01 kibana[1056]: [2024-06-19T14:50:26.451+01:00][INFO ][plugins.alerting] Registering resources for context "security".
Jun 19 14:50:26 smg-lsm-m01 kibana[1056]: [2024-06-19T14:50:26.514+01:00][INFO ][plugins.assetManager] Server is NOT enabled
Jun 19 14:50:26 smg-lsm-m01 kibana[1056]: [2024-06-19T14:50:26.522+01:00][INFO ][plugins.alerting] Registering resources for context "observability.apm".
Jun 19 14:50:27 smg-lsm-m01 kibana[1056]: [2024-06-19T14:50:27.036+01:00][ERROR][elasticsearch-service] Unable to retrieve version information from Elasticsearch nodes. certificate has expired
Jun 19 14:50:28 smg-lsm-m01 kibana[1056]: [2024-06-19T14:50:28.060+01:00][INFO ][plugins.screenshotting.chromium] Browser executable: /usr/share/kibana/node_modules/@kbn/screenshotting-plugin/chromium/headless_shell-linux_x64/headless_

2

Official Elastic Search 8.14 + Docker 4.30 Repo via JARaaS Hybrid RAG - Documentation - code current 6/19/2024 [AI Generated]

Note: Sources at the end of the response

Hi Team,

The main issue indicated by the log messages is that Kibana is unable to connect to Elasticsearch nodes due to an expired certificate:

Unable to retrieve version information from Elasticsearch nodes. certificate has expired

Steps to Resolve

  1. Renew or Replace the Expired Certificate:

    • Check which certificate has expired. Typically, this involves the CA (Certificate Authority) certificate, the server certificate, or the client certificate.
    • Generate a new certificate if needed using your certificate authority or renew the existing one.

  2. Update Elasticsearch and Kibana Configurations:

    • Update the Elasticsearch configuration (elasticsearch.yml) and Kibana configuration (kibana.yml) files with the new certificate paths.
    • Restart both Elasticsearch and Kibana services to apply the changes.

Example Configurations

Elasticsearch Configuration (elasticsearch.yml):

xpack.security.transport.ssl.keystore.path: "path_to_keystore.jks"
xpack.security.transport.ssl.truststore.path: "path_to_truststore.jks"

Kibana Configuration (kibana.yml):

server.ssl.enabled: true
server.ssl.certificate: path_to_new_certificate.crt
server.ssl.key: path_to_new_certificate.key
elasticsearch.ssl.certificateAuthorities: [ "path_to_CA.crt" ]

General Troubleshooting Tips

  • Ensure Matching Configuration: Validate that both Elasticsearch and Kibana have matching SSL/TLS settings.
  • Check Logs: Continue monitoring Kibana and Elasticsearch logs for any errors after making the changes.
  • Verify Connectivity: Use tools like curl to verify that you can establish a connection from Kibana to Elasticsearch over HTTPS.

Additional Information

For more detailed instructions on configuring SSL/TLS in Elasticsearch and Kibana, refer to their official documentation.

Sources:

  • Troubleshooting SAML: troubleshooting.asciidoc (internal document)
  • SAML Configuration Guide: saml-guide.asciidoc (internal document)

If you have any specific requirements or face issues during the process, feel free to reach out.

4

Hi Team, I have checked the lastic certificate and its not expire yet.

image
image1455×492 25.6 KB

image
image439×548 19 KB

image
image406×515 12.1 KB

Please advise if I'm not looking at the right certificate. Apologise for my question I have little knowledge with Elasticsearch.

Best Regards

Rasheed

5

Hi Team,

Our Elastis is complaining about certificate - I have checked all the cerficate assigned to elastic none of them are expired.

Please advise any help is greatly appreciated

io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Received fatal alert: bad_certificate
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:499) ~[?:?]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[?:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[?:?]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) ~[?:?]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:689) ~[?:?]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:652) ~[?:?]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) ~[?:?]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[?:?]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
at java.lang.Thread.run(Thread.java:1570) ~[?:?]
Caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert: bad_certificate
at sun.security.ssl.Alert.createSSLException(Alert.java:130) ~[?:?]
at sun.security.ssl.Alert.createSSLException(Alert.java:117) ~[?:?]
at sun.security.ssl.TransportContext.fatal(TransportContext.java:365) ~[?:?]
at sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:287) ~[?:?]
at sun.security.ssl.TransportContext.dispatch(TransportContext.java:204) ~[?:?]
at sun.security.ssl.SSLTransport.decode(SSLTransport.java:172) ~[?:?]
at sun.security.ssl.SSLEngineImpl.decode(SSLEngineImpl.java:736) ~[?:?]
at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:691) ~[?:?]
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:506) ~[?:?]
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:482) ~[?:?]
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:679) ~[?:?]
at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:310) ~[?:?]
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1445) ~[?:?]
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1338) ~[?:?]
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1387) ~[?:?]
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:529) ~[?:?]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:468) ~[?:?]

image
image786×349 11.6 KB

image
image722×362 11 KB

image
image1917×930 38 KB

6

Any help please is greatly appreciated