Kibana upgrade to 8.x , can't enter monitoring

Elastic Stack Kibana
1

Hi, i did the upgrade of Elasticsearch and Kibana to the latest version and ever since, i am not able to get into the Stack Monitoring webpage ::

image
image1107×294 21.5 KB

Here's the kibana.yml ::

server.host: "0.0.0.0"
server.publicBaseUrl: "https://cnq-elk.int.xxxxxxxxx.ca/"
server.name: "cnq-elk"
elasticsearch.hosts: ["https://127.0.0.1:9200"]
elasticsearch.username: "kibana_system"
elasticsearch.password: "xxxxxxxxxxxxxxxxxxxxxxx"
server.ssl.enabled: true
server.ssl.certificate: /etc/kibana/certs/kibana.pem
server.ssl.key: /etc/kibana/certs/kibana.key
elasticsearch.ssl.certificateAuthorities: [ "/etc/kibana/certs/selfca.pem" ]

And here my master node Elasticsearch.yml ::

cluster.name: cnq-elk
node.name: elk-node-1
node.roles: [ master,data ]
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: "0.0.0.0"
discovery.seed_hosts: ["127.0.0.1", "10.54.1.73"]
cluster.initial_master_nodes: ["elk-node-1"]
xpack.security.enabled: true
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: certs/elastic-certificates.p12
xpack.security.http.ssl.truststore.path: certs/elastic-certificates.p12
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12
cluster.max_shards_per_node: 10000

And finally, my data node Elasticsearch.yml ::

cluster.name: cnq-elk
node.name: elk-node-2
node.roles: [ data,remote_cluster_client,ingest  ]
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: ["0.0.0.0"]
discovery.seed_hosts: ["127.0.0.1", "10.54.1.72"]
cluster.initial_master_nodes: ["elk-node-1"]
xpack.security.enabled: false
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: certs/elastic-certificates.p12
xpack.security.http.ssl.truststore.path: certs/elastic-certificates.p12
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12
2

Hi @yhache !

You'll need to add the kibana_admin and monitoring_user roles to the user you've used to authenticate with kibana.

Configure security in Kibana | Kibana Guide [8.1] | Elastic has some information on how to update the roles for elastic stack users.

3

Thanks for the reply. I already gave all the access i could to the user. I even tried logging in with the "elastic" user. Nothing seems to be working.

4

That's strange. The Elastic user should definitely work.

That message comes from kibana/index.tsx at 9d662b77bec867247421376ecb60a07202ed45f9 · elastic/kibana · GitHub which is a page I find hard to reach even in local testing.

It calls api/monitoring/v1/check_access (kibana/check_access.ts at 0bd772432a313599656d8fe8adfdc7e6ef5d328a · elastic/kibana · GitHub) on kibana so it could be that API is failing for some other reason and the UI is treating it as an authorization error.

If an exception gets thrown there it should show up in your kibana logs when you try to load the monitoring UI, so I'd recommend checking there next.

5

Hi again @yhache !

I just came across Stack Monitoring "access denied" after 8.0.0 upgrade · Issue #125756 · elastic/kibana · GitHub which seems to be caused by newer versions of Elasticsearch requiring remote_cluster_client for CCS to work, which stack monitoring queries by default.

You may be able to work around the problem by either configuring that role for your Elasticsearch nodes or setting monitoring.ui.ccs.enabled: false in kibana if you don't need the Stack Monitoring UI to query via CCS.

6

I only had my data instance with the role remote_cluster_client . So i added it to the master node and did a service reset.

Everything is working now. Thanks @matschaffer

1 Like
7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.