Proxyconnect tcp: tls: first record does not look like a TLS handshake metricbeat monitoring elasticsearch

[Adriann (Adrian)]

Hello,

I wanted to set up beats monitoring for my enviroment. But it does not work and I see this error in the logs.

2020-10-28T14:25:23.636+0100	INFO	[monitoring]	elasticsearch/elasticsearch.go:245	Failed to connect to Elastic X-Pack Monitoring. Either Elasticsearch X-Pack monitoring is not enabled or Elasticsearch is not available. Will keep retrying. Error: cannot connect underlying Elasticsearch client: Get "https://dns.pl:9200": proxyconnect tcp: tls: first record does not look like a TLS handshake

This is how my metricbeat configuration looks like.

monitoring:
    enabled: true
    cluster_uuid: "someuuidtakenfromserver"
    elasticsearch:
        hosts: ["https://dns.pl:9200"]
        username: "${ES_LOG}"
        password: "${ES_PWD}"
        ssl.certificate_authorities: ["pathcerts/ca.crt"]
        ssl.certificate: "path/certs/beats.crt"
        ssl.key: "path/certs/beats.key"
        ssl.verification_mode: "none"
        proxy_disable: true	

This is how my monitoring cluster node looks like.

xpack.security.enabled: true
xpack.security.http.ssl.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.http.ssl.key: certs/node1/node1.key
xpack.security.http.ssl.certificate: certs/node1/node1.crt
xpack.security.http.ssl.certificate_authorities: certs/ca/ca.crt
xpack.security.transport.ssl.key: certs/node1/node1.key
xpack.security.transport.ssl.certificate: certs/node1/node1.crt
xpack.security.transport.ssl.certificate_authorities: certs/ca/ca.crt
xpack.security.transport.ssl.verification_mode: certificate
xpack.monitoring.enabled: true
xpack.monitoring.collection.enabled: true
xpack.security.audit.enabled: true
discovery.type: single-node
script.max_compilations_rate: 3000/5m
search.max_buckets: 80000
cluster.max_shards_per_node: 3000

Everything else is working.
Elasticsearch-pack from metricbeat that is on node server is working. Kibana monitoring is also working.

Roles are as below.

If I just do

monitoring.enabled: true
monitoring.cluster_uuid: someuid

on a beat that is on the same server as clutser node everything works.

What am I doing wrong?

EDITED:

I guess this issue is related to this
But what should I do If I configured xpack as in this thread?

[Adriann (Adrian)]

So after testing a lot of things for hours It just looks like the

proxy_disable: true

and

proxy_url : ""

does not behave like documented.

[Adriann (Adrian)]

After enabling beat-xpack and adding configuration to every beat
http.enabled: true
http.port: 506X
monitoring.enabled: false

to beats I got the data I wanted.

[Adriann (Adrian)]

This is still no perfect because it somehow rotates the beats info...

I have the same configuration on 6 servers. But the system sees only one kind of a beat and constantly changes the name.