# Error fetching data for metricset kibana.node\_rules: error making http request

**URL:** <https://discuss.elastic.co/t/error-fetching-data-for-metricset-kibana-node-rules-error-making-http-request/349990>\
**Category:** Elastic Observability\
**Created:** [December 27, 2023, 1:02am UTC](https://discuss.elastic.co/t/error-fetching-data-for-metricset-kibana-node-rules-error-making-http-request/349990 "2023-12-27T01:02:51Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![husoelasticbe](https://avatars.discourse-cdn.com/v4/letter/h/b5ac83/32.png) [@husoelasticbe](https://discuss.elastic.co/u/husoelasticbe)\
**Post date:** [December 27, 2023, 1:02am UTC](https://discuss.elastic.co/t/error-fetching-data-for-metricset-kibana-node-rules-error-making-http-request/349990/1 "2023-12-27T01:02:51Z")

</div>

Hi Folks,  
I am almost getting mad. Please help me our here. I am trying to collect kibana monitoring data with metricbeat. I get strangely the following error:

```auto
{"file.name":"module/wrapper.go","file.line":256},"message":"Error fetching data for metricset kibana.node_rules: error making http request: Get \"https://10.10.10.11:5601/api/status\. x509: certificate signed by unknown authority","service.name":"metricbeat","ecs.version":"1.6.0"}

{"file.name":"module/wrapper.go","file.line":256},"message":"Error fetching data for metricset kibana.node_actions: error making http request: Get \"https://10.10.10.11:5601/api/status. x509: certificate signed by unknown authority","service.name":"metricbeat","ecs.version":"1.6.0"}

metricset kibana.stats: error making http request: Get \"https://10.10.10.11:5601/api/status\": x509: certificate signed by unknown authority","service.name":"metricbeat","ecs.version":"1.6.0"}. x509: certificate signed by unknown authority","service.name":"metricbeat","ecs.version":"1.6.0"}

```

it is saying that unknow authority but I generated the certs with the self generated CA

here is the test output of metricbeat:

```auto
root@kibana:/etc/metricbeat# metricbeat test output
elasticsearch: https://10.10.10.9:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 10.10.10.9
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK

```

here is metricbeat.yml config:  
-kibana

```auto
 host: "https://10.10.10.11:5601"
  username: "user"
  password: "passw"

```

- elastic.output:

```auto
#hosts: ["10.10.10.9:9200", "10.10.10.10:9200"]
  hosts: ["10.10.10.9:9200"]

  # Protocol - either `http` (default) or `https`.
  protocol: "https"

  # Authentication credentials - either API key or username/password.
  #api_key: "id:api_key"
  username: "elastic"
  password: "huseyin"
  #ssl.certificate: "/etc/kibana/certs/chamber1/chamber1.crt"
  #ssl.key: "/etc/kibana/certs/chamber1/chamber1.key"
  ssl.certificate_authorities: "/etc/kibana/certs/elasticsearch

```

here is kibana.yml:

```auto
# =================== System: Elasticsearch ===================
# The URLs of the Elasticsearch instances to use for all your queries.
elasticsearch.hosts: ["https://10.10.10.9:9200", "https://10.10.10.10:9200"]

# If your Elasticsearch is protected with basic authentication, these settings provide
# the username and password that the Kibana server uses to perform maintenance on the Kibana
# index at startup. Your Kibana users still need to authenticate with Elasticsearch, which
# is proxied through the Kibana server.
elasticsearch.username: "user"
elasticsearch.password: "passw"
ssl.certificate: "/etc/kibana/certs/chamber1/chamber1.crt"
ssl.key: "/etc/kibana/certs/chamber1/chamber1.key"
ssl.certificate_authorities: "/etc/kibana/certs/elasticsearch-ca.pem"

```

here is the kibana authority cert:

```auto
elasticsearch.ssl.certificateAuthorities: ["/etc/kibana/certs/elasticsearch-ca.pem"]

```

cert generate command:

```auto
#Generate crt and key file: 
./bin/elasticsearch-certutil cert --pem --silent --in instances.yml --out certs.zip -ca /usr/share/elasticsearch/elastic-stack-ca.p12	

```

here is the modules.d/kibana-xpack.yml file:

```auto
module: kibana
  # metricsets:
  # - stats
  period: 10s
  xpack.enabled: true
  hosts: ["https://10.10.10.11:5601"]
  username: "kibana_huseyin"
  password: "huseyin"
  #basepath: ""
  server.ssl.enabled: true
  server.ssl.certificate: "/etc/kibana/certs/kibana-server.crt"
   server.ssl.key: "/etc/kibana/certs/kibana-server.key"
  #ssl.certificateAuthorities: ["/etc/kibana/certs/elasticsearch-ca.pem"]

```

Can you please help me with this issue? how can I test or verify that I have the same CA .  
I didnt see any section in the official side regarding, securing the kibana elasticsearch logstash with self generated CA and ssl certs.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 27, 2023, 1:58am UTC](https://discuss.elastic.co/t/error-fetching-data-for-metricset-kibana-node-rules-error-making-http-request/349990/2 "2023-12-27T01:58:04Z")

</div>

@husoelasticbe

Please share your entire kibana.yml

Use 3 backticks to format code.

What URL do you use to bring up kibana in browser?

Pretty sure you have this incorrect backwards

> [@husoelasticbe](#):
>
> ```auto
> #server.ssl.enabled: true
> #server.ssl.certificate: "/etc/kibana/certs/kibana-server.crt"
> #server.ssl.key: "/etc/kibana/certs/kibana-server.key"
> ssl.certificateAuthorities: ["/etc/kibana/certs/elasticsearch-ca.pem"]
> 
> ```

---

<div class="post-metadata">

**Author:** ![husoelasticbe](https://avatars.discourse-cdn.com/v4/letter/h/b5ac83/32.png) [@husoelasticbe](https://discuss.elastic.co/u/husoelasticbe)\
**Post date:** [December 27, 2023, 2:20am UTC](https://discuss.elastic.co/t/error-fetching-data-for-metricset-kibana-node-rules-error-making-http-request/349990/3 "2023-12-27T02:20:41Z")

</div>

> [@stephenb](#):
>
> use to bring up kibana in browser

here is the kibana.yml:

```auto
https://gist.github.com/husobaba/06939d8e5e57d6368e115b30d8633a94

```

here is the metricbeat:

```auto
https://gist.github.com/husobaba/34583472988f65917d04b5ed2463efe5

```

What URL do you use to bring up kibana in browser?  
[https://10.10.10.11:5601/](https://10.10.10.11:5601/)

I hope you can check the yml files with gist.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 27, 2023, 2:38am UTC](https://discuss.elastic.co/t/error-fetching-data-for-metricset-kibana-node-rules-error-making-http-request/349990/4 "2023-12-27T02:38:03Z")

</div>

In kibana.yml

These are not needed / meaningless since they are not prefixed with `server` or `elasticsearch`

```auto
ssl.certificate: "/etc/kibana/certs/chamber1/chamber1.crt"
ssl.key: "/etc/kibana/certs/chamber1/chamber1.key"
ssl.certificate_authorities: "/etc/kibana/certs/elasticsearch-ca.pem"

```

In the module

Not sure why you configured that way... You commented out lines.. changed things..

Start over on that and just enable, put in the metricsets and put in the certificate authority setting and user password etc

Not other stuff...

Remember in the module is about connecting to Kibana

Please share when you fixed

---

<div class="post-metadata">

**Author:** ![husoelasticbe](https://avatars.discourse-cdn.com/v4/letter/h/b5ac83/32.png) [@husoelasticbe](https://discuss.elastic.co/u/husoelasticbe)\
**Post date:** [December 27, 2023, 3:42am UTC](https://discuss.elastic.co/t/error-fetching-data-for-metricset-kibana-node-rules-error-making-http-request/349990/5 "2023-12-27T03:42:10Z")

</div>

Ok I will try to take them out. and test that way. Nevertheless, Elasticsearch monitoring data is working fine. when this will be fixed I will share here as well.

---

<div class="post-metadata">

**Author:** ![husoelasticbe](https://avatars.discourse-cdn.com/v4/letter/h/b5ac83/32.png) [@husoelasticbe](https://discuss.elastic.co/u/husoelasticbe)\
**Post date:** [December 27, 2023, 3:20pm UTC](https://discuss.elastic.co/t/error-fetching-data-for-metricset-kibana-node-rules-error-making-http-request/349990/6 "2023-12-27T15:20:31Z")

</div>

> [@stephenb](#):
>
> lease share when yo

It is really frustrating, I am still not able to collecting monitoring data using metricbeat. Here is the way that I followed. (I elaborated all the steps that I have taken)  
Encrypt traffic between your browser and Kibana:

```auto
*** Kibana and the browser secure
cd /usr/share/elasticsearch
./bin/elasticsearch-certutil csr -name kibana-server 

mkdir CA_kibana
cp csr-bundle.zip CA_kibana/
cd CA_kibana/
unzip csr-bundle.zip 
/usr/share/elasticsearch/bin/elasticsearch-certutil cert --pem -ca /usr/share/elasticsearch/elastic-stack-ca.p12 -name kibana-server
this command will generate the certificate-bundle.zip
unzip that file and take the crt file and copy this to the kibana server. 

```

\*\*\*\*\* Change the following parameters:

```auto
server.ssl.enabled: true
server.ssl.certificate: /etc/kibana/certs/kibana-server.crt
server.ssl.key: /etc/kibana/certs/kibana-server.key

```

At this point my kibana and browser communications are in https.

I installed the metricbeat where my Elasticsearch is and I enabled the module elasticsearch-xpack to be able to collect monitoring data of elasticsearch.  
Here are the modifications:

Metricbeat.yml:

```auto
output.elasticsearch:
  # Array of hosts to connect to.
  #hosts: ["localhost:9200"]
  hosts: ["10.10.10.9:9200", "10.10.10.10:9200"]

  # Protocol - either `http` (default) or `https`.
  protocol: "https"

  # Authentication credentials - either API key or username/password.
  #api_key: "id:api_key"
  username: "elastic"
  password: "elastic"
  #ssl.certificate: "/usr/share/elasticsearch/huseyin/chamber1/chamber1.crt"
  #ssl.key: "/usr/share/elasticsearch/huseyin/chamber1/chamber1.key"
  ssl.certificate_authorities: "/usr/share/elasticsearch/old_certs/kibana/elasticsearch-ca.pem"
  

```

elasticsearch-xpack.yml file:

```auto
# Module: elasticsearch
# Docs: https://www.elastic.co/guide/en/beats/metricbeat/8.11/metricbeat-module-elasticsearch.html

- module: elasticsearch
  xpack.enabled: true
  period: 10s
  hosts: ["https://10.10.10.9:9200"]
  username: "elastic"
  password: "elastic"
  ssl.enabled: true  
  ssl.certificate_authorities: "/usr/share/elasticsearch/old_certs/kibana/elasticsearch-ca.pem" 

```

Until here everything is ok and smoothly configured. Additional info: The pem file: elasticsearch-ca.pem is coming from while Encrypting TLS on the HTTP client communications for Elasticsearch

```auto
command: ./bin/elasticsearch-certutil http

```

when I start to perform the same approach to collect kibana data I got the issue regarding :

```auto
error making http request: Get \"https://10.10.10.11:5601/api/status\": x509: cannot validate certificate for 10.10.10.11 because it doesn't contain any IP SANs","service.name

```

then when I try to generate new certificate wherein we can see the SAN (generating with my own CA)

```auto
command: ./bin/elasticsearch-certutil cert --pem --silent --in instances.yml --out certs.zip -ca /usr/share/elasticsearch/elastic-stack-ca.p12	

```

\*\*\*example instances.yml file:

```auto
instances: 
  - name: "chamber1"
	- ip:
	- "10.10.10.9"
  - name: "chamber2"
	- ip:
	- "10.10.10.10"
	  - name: "logstash"
	- ip:
	- "10.10.10.13"

```

then I added the newly generated certificate in the configuration files.  
metricbeat.yml:

```auto
setup.kibana:

  # Kibana Host
  # Scheme and port can be left out and will be set to the default (http and 5601)
  # In case you specify and additional path, the scheme is required: http://localhost:5601/path
  # IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
  host: "https://10.10.10.11:5601"
  ssl.enabled: true
  ssl.certificate_authorities: "/etc/kibana/certs/kibana5/kibana.crt"
  username: "kibana"
  password: "kibana"
  
 output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["10.10.10.9:9200"]

  # Protocol - either `http` (default) or `https`.
  protocol: "https"

  # Authentication credentials - either API key or username/password.
  #api_key: "id:api_key"
  username: "elastic"
  password: "elastic"  
  ssl.certificate_authorities: "/etc/kibana/certs/kibana5/kibana.crt"

```

kibana-xpack.yml:

```auto

# Module: kibana
# Docs: https://www.elastic.co/guide/en/beats/metricbeat/8.11/metricbeat-module-kibana.html

- module: kibana
  #metricsets:
  # - stats
  period: 10s
  xpack.enabled: true
  hosts: ["https://10.10.10.11:5601"]
  username: "kibana"
  password: "kibana"
  ssl.enabled: true
  ssl.certificate_authorities: "/etc/kibana/certs/kibana5/kibana.crt"
  

```

Then I get the following error:  
error making http request: Get "[https://10.10.10.11:5601/api/status\](https://10.10.10.11:5601/api/status%5C)": x509: certificate signed by unknown authority","service.name":"metricbeat","ecs.version":"1.6.0"}

While I'm generating the cert I used my own generated CA. Thus, do you have an idea where I am making a mistake? It will be very helpful to understand this procedure regarding secure collecting monitoring data of Kibana with metricbeat. Because this is not writing anywhere in the official website or at least I am missing it.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 27, 2023, 4:19pm UTC](https://discuss.elastic.co/t/error-fetching-data-for-metricset-kibana-node-rules-error-making-http-request/349990/7 "2023-12-27T16:19:41Z")

</div>

@husoelasticbe Sorry it is frustrating but this is really all cert stuff, not that different then any other web server etc...

BTW you have muted this topic.

You are generating your own certs via `csr` see [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/certutil.html#certutil-csr) so the assumption is that you have a working knowledge of certs and you have an actual certificate authority that you are submitting your `csr`s to. Are you intentionally using `csr` ?  
If you are NOT using an actual `csr` you should not be using `csr` mode.

> The `csr` mode generates certificate signing requests (CSRs) that you can send to a trusted certificate authority to obtain signed certificates.

In general this is SSL Cert Stuff.... Generate a CA then Generate Certs.  
Where the settings ask for CAs ... use CAs  
Where the setting ask for certs ... use Certs  
You seem to be confusing these two.  
Also in sections where the code is communicating with Elastic ... use the elastic CAs' / Certs  
in sections where the code is communicating with Kibana ... use the Kibana CAs' / Certs  
And you can use the same CA for all if you want.

Overall I would get Elasticsearch and Kibana working... make sure it is working...  
Then setup metricbeat

> [@husoelasticbe](#):
>
> metricbeat.yml:
> 
> ```auto
> setup.kibana:
> 
> # Kibana Host
> # Scheme and port can be left out and will be set to the default (http and 5601)
> # In case you specify and additional path, the scheme is required: http://localhost:5601/path
> # IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
> host: "https://10.10.10.11:5601"
> ssl.enabled: true
> ssl.certificate_authorities: "/etc/kibana/certs/kibana5/kibana.crt" 
> ^^^^ Is this the CA that you generated the Kibana cert from? does not look like it
> THIS Section is communicating with Kibana 
> username: "kibana"
> password: "kibana"
>   
> output.elasticsearch:
> # Array of hosts to connect to.
> hosts: ["10.10.10.9:9200"]
> 
> # Protocol - either `http` (default) or `https`.
> protocol: "https"
> 
> # Authentication credentials - either API key or username/password.
> #api_key: "id:api_key"
> username: "elastic"
> password: "elastic"  
> ssl.certificate_authorities: "/etc/kibana/certs/kibana5/kibana.crt" 
> ^^^^ Is this the CA that you generated the Elasticsearc cert from? does not look like it
> THIS Section is communicating with Elasticsearch 
> 
> ```

---

<div class="post-metadata">

**Author:** ![husoelasticbe](https://avatars.discourse-cdn.com/v4/letter/h/b5ac83/32.png) [@husoelasticbe](https://discuss.elastic.co/u/husoelasticbe)\
**Post date:** [December 27, 2023, 4:21pm UTC](https://discuss.elastic.co/t/error-fetching-data-for-metricset-kibana-node-rules-error-making-http-request/349990/8 "2023-12-27T16:21:17Z")

</div>

I didnt know it. I changed it to watching

---

<div class="post-metadata">

**Author:** ![husoelasticbe](https://avatars.discourse-cdn.com/v4/letter/h/b5ac83/32.png) [@husoelasticbe](https://discuss.elastic.co/u/husoelasticbe)\
**Post date:** [December 27, 2023, 5:55pm UTC](https://discuss.elastic.co/t/error-fetching-data-for-metricset-kibana-node-rules-error-making-http-request/349990/9 "2023-12-27T17:55:06Z")

</div>

> [@stephenb](#):
>
> If you are NOT using an actual `csr` you should not be using `cs`

I will redo everything and I will write everything step by step.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 27, 2023, 6:03pm UTC](https://discuss.elastic.co/t/error-fetching-data-for-metricset-kibana-node-rules-error-making-http-request/349990/10 "2023-12-27T18:03:21Z")

</div>

Let's back up a bit.  
What are you trying to accomplish? And I mean that from a cert/SSL perspective

If you just install from scratch and the the stack Auto configure security it will set everything up for you with the exception of https on Kibana.

I highly recommend reading this section in detail. Walk through the sections....

> **[Manually configure security | Elasticsearch Guide \[8.11\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/manually-configure-security.html)**

And even for the Kibana setup

> **[Set up basic security for the Elastic Stack plus secured HTTPS traffic |...](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-basic-setup-https.html#encrypt-kibana-browser)**

---

<div class="post-metadata">

**Author:** ![husoelasticbe](https://avatars.discourse-cdn.com/v4/letter/h/b5ac83/32.png) [@husoelasticbe](https://discuss.elastic.co/u/husoelasticbe)\
**Post date:** [December 27, 2023, 8:15pm UTC](https://discuss.elastic.co/t/error-fetching-data-for-metricset-kibana-node-rules-error-making-http-request/349990/11 "2023-12-27T20:15:15Z")

</div>

I look those pages and I made all the configuration again, now I can monitor them with metricbeat.  
I made the exactly the same steps: (again).

When I setup the basic security, I used the commands

generate ca:

./bin/elasticsearch-certutil ca

generate the certificate and private key:

./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12

encrytion of internode communications with TLS/SSL:

I used the elastic-certificates.p12 which is previous command output.

this way my elasticsearch nodes are secured TLS/SSL.

next step was "Set up basic security for the Elastic Stack plus secured HTTPS traffic"

Encrypt HTTP client communications for Elasticsearch. I used the command (like kibana to elasticsearch) ===\>

./bin/elasticsearch-certutil http

like you mentioned:

```auto
Also in sections where the code is communicating with Elastic ... use the elastic CAs' / Certs
in sections where the code is communicating with Kibana ... use the Kibana CAs' / Certs
And you can use the same CA for all if you want.

```

And I used the same CA for all, Thank you for your guidance Stephen

While I am reading the sections, I also saw that we can also collect data with elastic agent after the version 8.5. Even with Fleet, we can centralize all those agents and their policies, we can check them they are healthy or not. It sounds great to me!  
Currently I have 8.11 so I will try to set this up and see the differences. Eventually I can share my findings here.  
Once again thank you for your help.