Trying to get my docker-compose to start Kibana with ssl

Elastic Stack Kibana
1

Hi All

I'm trying to set up my kibana to use SSL for 7.16.3, like I set up Elasticsearch.
So I used this guide : Encrypting communications in an Elasticsearch Docker Container | Elasticsearch Guide [7.17] | Elastic
to set up Elasticsearch.
I then expanded the instances.yml to create certs for kibana
Looking int the 'certs' disc I see these folders (and zip file):
bundle.zip ca elastichq es01 es02 es03 kibana logstash
and inside the kibana folder I see
kibana.crt kibana.key
I then set up docker-compose.yml like this:

  kibana:
    image: docker.elastic.co/kibana/kibana:7.16.3
    container_name: kibana
    environment:
      - SERVER_NAME="kibana.onead.dk"
      - node.name=kibana
      - ELASTICSEARCH_HOSTS="http://elasticsearch:9200"
      - XPACK_SECURITY_ENABLED=true
      - ELASTICSEARCH_USERNAME="kibana"
      - ELASTICSEARCH_PASSWORD="XpYnSc0K4ziDuryfkMnH"
      - ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES="$CERTS_DIR/kibana/kibana.crt"
      - ELASTICSEARCH_SSL_VERIFICATIONMODE=certificate
      - SERVER_SSL_ENABLED=true
      - SERVER_SSL_KEY="$CERTS_DIR/kibana/kibana.key"
      - SERVER_SSL_CERTIFICATE="$CERTS_DIR/kibana/kibana.crt"
    ulimits:
      nproc: 65535
      memlock:
        soft: -1
        hard: -1
    cap_add:
      - ALL

Which is the same path for the certs as set in the Elasticsearch.

But looking in the logs I see this message:

kibana            | {"type":"log","@timestamp":"2022-02-08T09:29:28+00:00","tags":["fatal","root"],"pid":8,"message":"Error: ENOENT: no such file or directory, open '/usr/share/elasticsearch/config/certificates/kibana/kibana.key'\n    at Object.openSync (node:fs:585:3)\n    at readFileSync (node:fs:453:35)\n    at readFile (/usr/share/kibana/node_modules/@kbn/server-http-tools/target_node/ssl/ssl_config.js:175:47)\n    at new SslConfig (/usr/share/kibana/node_modules/@kbn/server-http-tools/target_node/ssl/ssl_config.js:126:18)\n    at new HttpConfig (/usr/share/kibana/src/core/server/http/http_config.js:256:16)\n    at MapSubscriber.project (/usr/share/kibana/src/core/server/http/http_service.js:62:239)\n    at MapSubscriber._next (/usr/share/kibana/node_modules/rxjs/internal/operators/map.js:49:35)\n    at MapSubscriber.Subscriber.next (/usr/share/kibana/node_modules/rxjs/internal/Subscriber.js:66:18)\n    at CombineLatestSubscriber.notifyNext (/usr/share/kibana/node_modules/rxjs/internal/observable/combineLatest.js:97:34)\n    at InnerSubscriber._next (/usr/share/kibana/node_modules/rxjs/internal/InnerSubscriber.js:28:21)\n    at InnerSubscriber.Subscriber.next (/usr/share/kibana/node_modules/rxjs/internal/Subscriber.js:66:18)\n    at MapSubscriber._next (/usr/share/kibana/node_modules/rxjs/internal/operators/map.js:55:26)\n    at MapSubscriber.Subscriber.next (/usr/share/kibana/node_modules/rxjs/internal/Subscriber.js:66:18)\n    at DistinctUntilChangedSubscriber._next (/usr/share/kibana/node_modules/rxjs/internal/operators/distinctUntilChanged.js:69:30)\n    at DistinctUntilChangedSubscriber.Subscriber.next (/usr/share/kibana/node_modules/rxjs/internal/Subscriber.js:66:18)\n    at MapSubscriber._next (/usr/share/kibana/node_modules/rxjs/internal/operators/map.js:55:26) {\n  errno: -2,\n  syscall: 'open',\n  code: 'ENOENT',\n  path: '/usr/share/elasticsearch/config/certificates/kibana/kibana.key'\n}"}
kibana            |
kibana            |  FATAL  Error: ENOENT: no such file or directory, open '/usr/share/elasticsearch/config/certificates/kibana/kibana.key'
kibana            |
kibana exited with code 1

So for me it looks like it's trying to set up the certificates, but doesn't get the path? How do I fix that?

2

Oh darned, I had forgotten to add the volume to kibana holding the certs, so after doing that, it starts, I also changed the http to https of course.

I now see other messages:

kibana            | {"type":"log","@timestamp":"2022-02-08T09:55:00+00:00","tags":["warning","plugins","security","config"],"pid":8,"message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command."}
kibana            | {"type":"log","@timestamp":"2022-02-08T09:55:05+00:00","tags":["error","elasticsearch-service"],"pid":8,"message":"Unable to retrieve version information from Elasticsearch nodes. unable to verify the first certificate"}

So, one step closer...

3

Ok, so I added an encryption key and also tried to ask it to not verify the certificate by adding two config lines:

  kibana:
    image: docker.elastic.co/kibana/kibana:7.16.3
    container_name: kibana
    environment:
      - SERVER_NAME=kibana.onead.dk
      - node.name=kibana
      - ELASTICSEARCH_HOSTS="https://elasticsearch:9200"
      - XPACK_SECURITY_ENABLED=true
      - ELASTICSEARCH_USERNAME=kibana
      - ELASTICSEARCH_PASSWORD=XpYnSc0K4ziDuryfkMnH
      - ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=$CERTS_DIR/kibana/kibana.crt
      - ELASTICSEARCH_SSL_VERIFICATIONMODE=certificate
      - SERVER_SSL_ENABLED=true
      - SERVER_SSL_KEY=$CERTS_DIR/kibana/kibana.key
      - SERVER_SSL_CERTIFICATE=$CERTS_DIR/kibana/kibana.crt
      - xpack.encryptedSavedObjects.encryptionKey="9XeNMpw3EYfxst2XhHE5gj3GRr22k9y59KkYX6DD"
      - xpack.reporting.capture.browser.chromium.disableSandbox=true
      - xpack.security.transport.ssl.verification_mode=none
    ulimits:
      nproc: 65535
      memlock:
        soft: -1
        hard: -1
    cap_add:
      - ALL
    volumes:
      - certs:$CERTS_DIR

But unfortunately it doesn't see the three last lines, the xpack ones, it still complains about the same?

4

And going to the website it says 'Kibana is not ready yet" ??

5

Ok, I got some information from the logfile:

kibana            | {"type":"log","@timestamp":"2022-02-08T13:36:28+00:00","tags":["info","plugins-service"],"pid":8,"message":"Plugin \"metricsEntities\" is disabled."}
kibana            | {"type":"log","@timestamp":"2022-02-08T13:36:29+00:00","tags":["info","http","server","Preboot"],"pid":8,"message":"http server running at https://0.0.0.0:5601"}
kibana            | {"type":"log","@timestamp":"2022-02-08T13:36:29+00:00","tags":["warning","config","deprecation"],"pid":8,"message":"Starting in 8.0, the Kibana logging format will be changing. This may affect you if you are doing any special handling of your Kibana logs, such as ingesting logs into Elasticsearch for further analysis. If you are using the new logging configuration, you are already receiving logs in both old and new formats, and the old format will simply be going away. If you are not yet using the new logging configuration, the log format will change upon upgrade to 8.0. Beginning in 8.0, the format of JSON logs will be ECS-compatible JSON, and the default pattern log format will be configurable with our new logging system. Please refer to the documentation for more information about the new logging format."}
kibana            | {"type":"log","@timestamp":"2022-02-08T13:36:29+00:00","tags":["warning","config","deprecation"],"pid":8,"message":"Kibana is configured to authenticate to Elasticsearch with the \"kibana\" user. Use a service account token instead."}
kibana            | {"type":"log","@timestamp":"2022-02-08T13:36:29+00:00","tags":["warning","config","deprecation"],"pid":8,"message":"Use Kibana application privileges to grant reporting privileges. Using  \"xpack.reporting.roles.allow\" to grant reporting privileges is deprecated. The \"xpack.reporting.roles.enabled\" setting will default to false in a future release."}
kibana            | {"type":"log","@timestamp":"2022-02-08T13:36:29+00:00","tags":["warning","config","deprecation"],"pid":8,"message":"Enabling or disabling the Security plugin in Kibana is deprecated. Configure security in Elasticsearch instead."}
kibana            | {"type":"log","@timestamp":"2022-02-08T13:36:29+00:00","tags":["warning","config","deprecation"],"pid":8,"message":"User sessions will automatically time out after 8 hours of inactivity starting in 8.0. Override this value to change the timeout."}
kibana            | {"type":"log","@timestamp":"2022-02-08T13:36:29+00:00","tags":["warning","config","deprecation"],"pid":8,"message":"Users are automatically required to log in again after 30 days starting in 8.0. Override this value to change the timeout."}
kibana            | {"type":"log","@timestamp":"2022-02-08T13:36:29+00:00","tags":["info","plugins-system","standard"],"pid":8,"message":"Setting up [113] plugins: [translations,licensing,globalSearch,globalSearchProviders,features,licenseApiGuard,code,usageCollection,xpackLegacy,taskManager,telemetryCollectionManager,telemetryCollectionXpack,kibanaUsageCollection,share,embeddable,uiActionsEnhanced,screenshotMode,banners,telemetry,newsfeed,mapsEms,mapsLegacy,kibanaLegacy,fieldFormats,expressions,dataViews,charts,esUiShared,bfetch,data,savedObjects,presentationUtil,expressionShape,expressionRevealImage,expressionRepeatImage,expressionMetric,expressionImage,customIntegrations,home,searchprofiler,painlessLab,grokdebugger,management,watcher,licenseManagement,advancedSettings,spaces,security,savedObjectsTagging,reporting,canvas,lists,ingestPipelines,fileUpload,encryptedSavedObjects,dataEnhanced,cloud,snapshotRestore,eventLog,actions,alerting,triggersActionsUi,transform,stackAlerts,ruleRegistry,visualizations,visTypeXy,visTypeVislib,visTypeVega,visTypeTimelion,visTypeTagcloud,visTypeTable,visTypePie,visTypeMetric,visTypeMarkdown,tileMap,regionMap,expressionTagcloud,expressionMetricVis,console,graph,fleet,indexManagement,remoteClusters,crossClusterReplication,indexLifecycleManagement,dashboard,maps,dashboardMode,dashboardEnhanced,visualize,visTypeTimeseries,rollup,indexPatternFieldEditor,lens,cases,timelines,discover,osquery,observability,discoverEnhanced,dataVisualizer,ml,uptime,securitySolution,infra,upgradeAssistant,monitoring,logstash,enterpriseSearch,apm,savedObjectsManagement,indexPatternManagement]"}
kibana            | {"type":"log","@timestamp":"2022-02-08T13:36:29+00:00","tags":["info","plugins","taskManager"],"pid":8,"message":"TaskManager is identified by the Kibana UUID: 9a05faa3-056a-41e2-ac32-ce208bf4df3e"}
kibana            | {"type":"log","@timestamp":"2022-02-08T13:36:29+00:00","tags":["warning","plugins","security","config"],"pid":8,"message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command."}
kibana            | {"type":"log","@timestamp":"2022-02-08T13:36:29+00:00","tags":["warning","plugins","security","config"],"pid":8,"message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command."}
kibana            | {"type":"log","@timestamp":"2022-02-08T13:36:29+00:00","tags":["warning","plugins","reporting","config"],"pid":8,"message":"Generating a random key for xpack.reporting.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.reporting.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command."}
kibana            | {"type":"log","@timestamp":"2022-02-08T13:36:29+00:00","tags":["info","plugins","ruleRegistry"],"pid":8,"message":"Installing common resources shared between all indices"}
kibana            | {"type":"log","@timestamp":"2022-02-08T13:36:34+00:00","tags":["error","elasticsearch-service"],"pid":8,"message":"Unable to retrieve version information from Elasticsearch nodes. connect ECONNREFUSED 192.168.160.3:9200"}
kibana            | {"type":"log","@timestamp":"2022-02-08T13:36:40+00:00","tags":["error","elasticsearch-service"],"pid":8,"message":"Unable to retrieve version information from Elasticsearch nodes. unable to verify the first certificate"}

So I'm told that I shouldn't use the kibana user to authenticate to Elasticsearch, I should use service account token instead.

I'm getting very confused, so the username / passwords generated in the guide mentioned in the beginning is not for this???

I'm getting stuck in this, I really need some help, please tell me what I'm doing wrong here...

6

Ok, a bit more progress, now I at least got the token created by running this inside the:

Elasticsearch-service-tokens create elastic/kibana kibana
SERVICE_TOKEN elastic/kibana/kibana = AA...

Now I'm trying to figure out how to give this in environment variables in docker-compose....

1 Like
7

Same problem here ... There is no documentation whatsoever of this ...

8

I created a new one with this guide:

And that worked nicely, now I can finally try to get my backup restored.

1 Like
9

Hey @fribse, could you document how you went about this?

Currently using Docker-Compose, would creating the token with Elasticsearch and then adding it to the docker-compose.yml for kibana fix the issue?

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.