# Multiple hosts per monitor confuses certificate validation since 7.17.0

**URL:** https://discuss.elastic.co/t/multiple-hosts-per-monitor-confuses-certificate-validation-since-7-17-0/296062
**Category:** Beats
**Tags:** heartbeat
**Created:** [February 2, 2022, 12:11pm UTC](https://discuss.elastic.co/t/multiple-hosts-per-monitor-confuses-certificate-validation-since-7-17-0/296062 "2022-02-02T12:11:40Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![PayBas](https://avatars.discourse-cdn.com/v4/letter/p/c57346/32.png) [@PayBas](https://discuss.elastic.co/u/PayBas)
#### Post date: [February 2, 2022, 12:11pm UTC](https://discuss.elastic.co/t/multiple-hosts-per-monitor-confuses-certificate-validation-since-7-17-0/296062/1 "2022-02-02T12:11:41Z")

</div>

Since upgrading Heartbeat to 7.17.0 all my monitors have gone haywire.

It seems that if you define multiple hosts per monitor like so:

```auto
- type: http
  id: cicd
  name: CI/CD
  hosts:
    - https://a.corp.internal
    - https://b.corp.internal
    - https://c.corp.internal
  check.response.status: [200]
  max_redirects: 3
  schedule: '@every 30s'

```

That you will get issues like:

```auto
io:Get "https://a.corp.internal": x509: certificate is valid for a.corp.internal, not b.corp.internal

```

This happens for each monitor where there are multiple hosts, with differing FQDNs.  
Monitors with a single host, or monitors with multiple hosts on the same FQDN, don't have this issue.

My setup was working fine for months until I updated from 7.16.3 to 7.17.0 today.

Update: couldn't find the cause of the issue. Reverting back to 7.16.3 worked.

---

<div class="post-metadata">

### Author: ![Andrew\_Cholakian1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrew_cholakian1/32/3612_2.png) [@Andrew\_Cholakian1](https://discuss.elastic.co/u/Andrew_Cholakian1)
#### Post date: [February 3, 2022, 9:20pm UTC](https://discuss.elastic.co/t/multiple-hosts-per-monitor-confuses-certificate-validation-since-7-17-0/296062/2 "2022-02-03T21:20:43Z")

</div>

Sorry to hear you're hitting this issue. It's a tricky one to debug because I'm having trouble replicating it. I tried to do so with the following config:

```auto
- type: http
  id: cicd
  name: CI/CD
  hosts:
    - https://elastic.co
    - https://google.com
  check.response.status: [200]
  max_redirects: 3
  schedule: '@every 30s'

```

however, that all seemed to work.

Can you replicate this behavior against any public sites so that we could reproduce it? The strange thing here is that 7.17.0 doesn't contain any changes that should impact this AFAIK.

Reading through the error it sounds like somehow heartbeat is mixing up the cert for one endpoint with that of another, however, I would think my attempt at replication would reveal that same issue.

---

<div class="post-metadata">

### Author: ![PayBas](https://avatars.discourse-cdn.com/v4/letter/p/c57346/32.png) [@PayBas](https://discuss.elastic.co/u/PayBas)
#### Post date: [February 4, 2022, 10:25am UTC](https://discuss.elastic.co/t/multiple-hosts-per-monitor-confuses-certificate-validation-since-7-17-0/296062/3 "2022-02-04T10:25:52Z")

</div>

I'll need some time to create a test setup to reproduce it. This was our production system which I had to roll back. I'll get back to you next week.

---

<div class="post-metadata">

### Author: ![PayBas](https://avatars.discourse-cdn.com/v4/letter/p/c57346/32.png) [@PayBas](https://discuss.elastic.co/u/PayBas)
#### Post date: [February 8, 2022, 12:55am UTC](https://discuss.elastic.co/t/multiple-hosts-per-monitor-confuses-certificate-validation-since-7-17-0/296062/4 "2022-02-08T00:55:25Z")

</div>

I have been able to reproduce the exact same behavior with a brand new setup. Different host, brand new containers, volumes, etc.

```auto
- type: http
  id: cicd
  name: CI/CD
  hosts:
    - https://bitbucket.corp.internal/status
    - https://jira.corp.internal/status
  check.response.status: [200]
  max_redirects: 3
  schedule: '@every 30s'
  ssl:
    certificate_authorities:
    - /etc/pki/ca-trust/source/anchors/corp-ca-bundle.pem
    supported_protocols:
    - TLSv1.1
    - TLSv1.2

```

All it takes to trigger the behavior is to have a minimum of 2 FQDN (in the same `monitor`) which _share a domain+TLD_ but have their own (non-wildcard) certificates.  
The reports will alternate between:

```auto
io:Get "https://jira.corp.internal/status": x509: 
certificate is valid for jira.corp.internal, not bitbucket.corp.internal

```

and

```auto
io:Get "https://bitbucket.corp.internal/status": x509: 
certificate is valid for bitbucket.corp.internal, not jira.corp.internal

```

So you are right, heartbeat (or elastic) is mixing up the certs between endpoints. And its not constant either. It's alternating in some unknown fashion.  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/8/3/83b62bdd5f341303078bbed299c935a77c5b4911.png)

Its very hard for me to provide an example with open-internet URL's, since my coporate network has an TLS interceptor, which obfuscates a lot.

But perhaps this would work for you (provided both have their own certificates, not shared wildcard cert).

```auto
  hosts:
    - https://discuss.elastic.co/
    - https://community.elastic.co/

```

Like I said. They have to share the same domain+TLD for the problem to occur.  
Using heartbeat 7.16.3 instead of 7.17.0 immediately fixes the issue (with the Elasticsearch version being constant at 7.17.0).

---

<div class="post-metadata">

### Author: ![TiagoQueiroz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tiagoqueiroz/32/107061_2.png) [@TiagoQueiroz](https://discuss.elastic.co/u/TiagoQueiroz)
#### Post date: [February 8, 2022, 5:23pm UTC](https://discuss.elastic.co/t/multiple-hosts-per-monitor-confuses-certificate-validation-since-7-17-0/296062/5 "2022-02-08T17:23:26Z")

</div>

@PayBas I managed to reproduce the issue as you described, it looks to be a very specific edge case that involves domains with a common suffix and using non-wildcard certificates.

I found two workarounds:

1. Set `ssl.verificaton_mode: certificate`

```auto
- type: http
  id: cicd
  name: CI/CD
  enabled: true
  urls:
    - https://bitbucket.corp.internal
    - https://jira.corp.internal
  check.response.status: [200]
  max_redirects: 3
  schedule: '@every 5s'
  ssl:
    certificate_authorities:
      - /home/tiago/devel/certGen/rootCA.crt
    verification_mode: certificate

```

1. Use different `monitors` for hosts that share a suffix and use non-wildcard certificates (in other words: hosts that cannot share a TLS certificate)

```auto
- type: http
  id: jira
  name: Jira
  enabled: true
  urls:
    - https://jira.corp.internal
  check.response.status: [200]
  max_redirects: 3
  schedule: '@every 5s'
  ssl:
    certificate_authorities:
      - /home/tiago/devel/certGen/rootCA.crt

- type: http
  id: bitbucket
  name: Bitbucket
  enabled: true
  urls:
    - https://bitbucket.corp.internal
  check.response.status: [200]
  max_redirects: 3
  schedule: '@every 5s'
  ssl:
    certificate_authorities:
      - /home/tiago/devel/certGen/rootCA.crt

```

---

<div class="post-metadata">

### Author: ![PayBas](https://avatars.discourse-cdn.com/v4/letter/p/c57346/32.png) [@PayBas](https://discuss.elastic.co/u/PayBas)
#### Post date: [February 8, 2022, 6:52pm UTC](https://discuss.elastic.co/t/multiple-hosts-per-monitor-confuses-certificate-validation-since-7-17-0/296062/6 "2022-02-08T18:52:57Z")

</div>

@TiagoQueiroz thank you for confirming my issue.

Workaround 2 (splitting monitors) is not practical for me. This would make my monitor files hundreds of lines long :). But I'll definitely look into `ssl.verificaton_mode: certificate`.

Otherwise I'll stick with 7.16.3 for now. The fact that I appear to be the only one running into this 7.17.0 bug suggests that it is indeed an edge case (one which hopefully will be addressed in a future release).

My question now is: do you guys create a ticket at [https://github.com/elastic/beats/issues?q=is%3Aopen+is%3Aissue+label%3AHeartbeat](https://github.com/elastic/beats/issues?q=is%3Aopen+is%3Aissue+label%3AHeartbeat) or should I?

---

<div class="post-metadata">

### Author: ![Andrew\_Cholakian1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrew_cholakian1/32/3612_2.png) [@Andrew\_Cholakian1](https://discuss.elastic.co/u/Andrew_Cholakian1)
#### Post date: [February 8, 2022, 10:17pm UTC](https://discuss.elastic.co/t/multiple-hosts-per-monitor-confuses-certificate-validation-since-7-17-0/296062/7 "2022-02-08T22:17:12Z")

</div>

I've opened an issue here [https://github.com/elastic/beats/issues/30290](https://github.com/elastic/beats/issues/30290) with additional thoughts. Let's continue the conversation here. Thanks @PayBas and @TiagoQueiroz for investigating here

---

<div class="post-metadata">

### Author: ![TiagoQueiroz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tiagoqueiroz/32/107061_2.png) [@TiagoQueiroz](https://discuss.elastic.co/u/TiagoQueiroz)
#### Post date: [February 9, 2022, 2:09pm UTC](https://discuss.elastic.co/t/multiple-hosts-per-monitor-confuses-certificate-validation-since-7-17-0/296062/8 "2022-02-09T14:09:46Z")

</div>

@PayBas Here is the fix: [https://github.com/elastic/beats/pull/30305](https://github.com/elastic/beats/pull/30305)

In the end it was a race condition, it seems to only happen when the hosts cannot share certs and the reply is very quick.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 9, 2022, 4:10pm UTC](https://discuss.elastic.co/t/multiple-hosts-per-monitor-confuses-certificate-validation-since-7-17-0/296062/9 "2022-03-09T16:10:00Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
