[MY_DOMAIN] = My externally accessable url such as "asdf.dk"
[MY_LOCAL_SERVER_IP] = My servers local IP address such as 192.168.1.2
Hello everyone,
I have some issues with kibana & certbot certificates.
Let me elaborate; I have generated certificates via certbot via the command
sudo certbot certonly --standalone
And followed this guide here for how to setup the permissions.
I am using nginx with kibana and elasticsearch.
From my desktop (which isn't the hosting machine) I can connect to https://[MY_DOMAIN]:9200. It will ask for credentials and if I provide them I see a json object like so:
{
"name" : "[MY_NODE_NAME]",
"cluster_name" : "[MY_CLUSTER_NAME]",
"cluster_uuid" : "[CLUSTER_UUID]",
"version" : {
"number" : "7.14.0",
"build_flavor" : "default",
"build_type" : "deb",
"build_hash" : "[BUILD_HASH]",
"build_date" : "2021-07-29T20:49:32.864135063Z",
"build_snapshot" : false,
"lucene_version" : "8.9.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
Which indicates I can connect to the elasticsearch server via https without issue.
However, when I try to start kibana.service it won't start. The errors I receive are the following:
{"type":"log","@timestamp":"2021-08-23T19:06:26+02:00","tags":["error","plugins","security","authentication"],
"pid":27369,"message":"License is not available, authentication is not possible."}
{"type":"log","@timestamp":"2021-08-23T19:06:26+02:00","tags":["warning","plugins","licensing"],
"pid":27369,"message":"License information could not be obtained from Elasticsearch due to ConnectionError: unable to get local issuer certificate error"}
also I think these are relevant:
{"type":"log","@timestamp":"2021-08-23T19:06:35+02:00","tags":
["warning","plugins","licensing"],"pid":27369,"message":
"License information could not be obtained from Elasticsearch
due to ConnectionError: unable to get local issuer certificate error"}
It seems to be that Kibana doesn't trust the certificate, but I have no clue as to what I should do to make it accept it. Please, could any of you more experienced people take a look at this and tell me where I've gone wrong? -Keep in mind I've tried to follow the guide in the aformentioned blogpost to the latter, without success.
Here's my Nginx config:
root@[myhost]:/etc/elasticsearch# cat /etc/nginx/sites-available/[MY_DOMAIN].conf
server {
listen [::]:80;
listen 80;
server_name [MY_DOMAIN];
return 301 https://[MY_DOMAIN]$request_uri;
}
server {
listen [::]:443 ssl http2;
listen 443 ssl http2;
server_name [MY_DOMAIN];
ssl_certificate /etc/letsencrypt/live/[MY_DOMAIN]/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/[MY_DOMAIN]/privkey.pem;
location / {
proxy_pass http://[MY_LOCAL_SERVER_IP]:9200;
proxy_redirect off;
proxy_read_timeout 90;
proxy_connect_timeout 90;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
}
} }
Here's my elasticsearch.yml:
transport.host: localhost
transport.tcp.port: 9300
http.port: 9200
network.host: 0.0.0.0
discovery.type: single-node
discovery.seed_hosts: [MY_DOMAIN] [MY_LOCAL_SERVER_IP]
cluster.name: [CLUSTER_NAME]
node.name: [NODE_NAME]
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.key: /etc/elasticsearch/certs/[MY_DOMAIN]/privkey2.pem
xpack.security.transport.ssl.certificate: /etc/elasticsearch/certs/[MY_DOMAIN]/cert2.pem
xpack.security.transport.ssl.certificate_authorities: [ "/etc/elasticsearch/certs/[MY_DOMAIN]/fullchain2.pem" ]
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.verification_mode: certificate
xpack.security.http.ssl.key: /etc/elasticsearch/certs/[MY_DOMAIN]/privkey2.pem
xpack.security.http.ssl.certificate: /etc/elasticsearch/certs/[MY_DOMAIN]/cert2.pem
And lastly my kibana.yml:
server.port: 5601
server.host: "[MY_LOCAL_SERVER_IP]"
elasticsearch.url: "https://[MY_DOMAIN]:9200"
elasticsearch.username: "[MY_USERNAME]"
elasticsearch.password: "[MY_PASSWORD]"
elasticsearch.hosts: ["https://[MY_LOCAL_SERVER_IP:9200"]
server.ssl.enabled: true
server.ssl.cert: /etc/kibana/ssl/[MY_DOMAIN]/fullchain2.pem
server.ssl.key: /etc/kibana/ssl/[MY_DOMAIN]/privkey2.pem
logging.verbose: true
Again, I can access the elasticsearch with my desktop without issue if I go via the browser or use curl https://[MY_DOMAIN]:9200 -u [MY_USERNAME]
[MY_PASSWORD]
However, if I try the same with via the server hosting everything, it won't work either. Curl also complains, saying:
root@[MY_LOCAL_SERVER_IP]:/etc/elasticsearch# curl https://[MY_DOMAIN]:9200
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
Do any of you guys have any idea what might be wrong?
Thank you so much in advance.
Kind regards, noobthenoob
EDITS: formatting of logs from one-liners to more readable format.