Connect Metricbeat to Elasticsearch 8.2.0 with password mode

[Dreinale (Enzo Brunet)]

HI,

I'm trying to connect metricbeat to my elasticsearch which I have configured yesterday with a password mode.
(https://www.youtube.com/watch?v=kkrLanotz1I&t=461s).
I have 2 users one is for elasticsearch (elastic) and kibana (kibana_system) with the same password (000).

Here is my elasticsearch.yml :

node.name: epnp
network.host: 192.168.66.11
http.port: 9200
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
cluster.initial_master_nodes: ["cluster"]
http.host: 0.0.0.0

kibana.yml :

server.port: 5601
server.host: "192.168.66.11"
server.publicBaseUrl: "http://192.168.66.11:5601"
elasticsearch.hosts: ["https://192.168.66.11:9200"]
elasticsearch.username: "kibana_system"
elasticsearch.password: "000"
elasticsearch.ssl.certificateAuthorities: [ "/etc/kibana/certs/http_ca.crt" ]

metricbeat.yml:

hosts: ["https://192.168.66.11:9200"]
username: "elastic"
password: "000"

So with that I tried in the metricbeat.yml

hosts: ["https://192.168.66.11:9200"] I got this error
x509: certificate signed by unknown authority

hosts: ["192.168.66.11:9200"] and I got this: couldn't connect to any of the configured Elasticsearch hosts.

So I'm stuck on this and I don't know how to resolve it if someone can help me it's can be really cool!

[stephenb (Stephen Brown)]

@Dreinale

hosts: ["https://192.168.66.11:9200"]
username: "elastic"
password: "000"

In the metricbeat.yml in the elasticsearch output section you need to provide a path the the CA for the elasticsearch self signed cert

See here

ssl.certificate_authorities: ["/etc/client/ca.pem"]

or you can set / turn off ssl verification (not recommended for production)

ssl.verification_mode : "none"

[Dreinale (Enzo Brunet)]

Thanks for the reply, i tried it but it's doesn't work I write on my metricbeat.yml :

output.elasticsearch:
  hosts: ["192.168.66.11:9200"]
  name: "elastic"
  password: "000"
  ssl.certificate_authorities: [ "/etc/elasticsearch/certs/http_ca.crt" ]

Exiting: couldn't connect to any of the configured Elasticsearch hosts

So i checked ur link and they need the:

output.elasticsearch.ssl.certificate: "/etc/client/cert.pem"
output.elasticsearch.ssl.key: "/etc/client/cert.key"

but I don't have cert except in my /etc/elasticsearch/certs i just have that :

http_ca.crt http.p12 transport.p12

Possibly i don't really understand ur reply but it's doesn't work.
Another tips ?

[stephenb (Stephen Brown)]

@Dreinale

output.elasticsearch:
  hosts: ["192.168.66.11:9200"]
  name: "elastic"
  password: "epnp000"
  ssl.certificate_authorities: [

You're missing a https So we'll never connect.

And no, you don't need both the cert and the pem

First, try the HTTPS with the verification equal to none and see if you can connect.

Then use the https with the CA

You may need to convert the CRT to a PEM

openssl x509 -in cert.crt -out cert.pem

[Dreinale (Enzo Brunet)]

Hum nothing works,

• https with ssl.verification_mode : "none" give me

Exiting: couldn't connect to any of the configured Elasticsearch hosts.

• the HTTP with ssl.certificate_authorities: [ "/etc/elasticsearch/certs/http_ca.crt" ]

Exiting: couldn't connect to any of the configured Elasticsearch hosts

• And I tried ssl.certificate_authorities: [ "/etc/elasticsearch/certs/cert.pem" ] after convert it and I got the same error...

notice that I always let the

  name: "elastic"
  password: "000"

[stephenb (Stephen Brown)]

hosts: ["192.168.66.11:9200"]

Needs to be

hosts: ["https://192.168.66.11:9200"]

From where you have metricbeat

curl --insecure -u "user: password" https://192.168.66.11:9200

[stephenb (Stephen Brown)]

Then perhaps you do not have connectivity between the metricbeat server and the Elasticsearch server. Are they on the same server?

Get the curl working first

[Dreinale (Enzo Brunet)]

Yeah it on the same server, but the curl doesn't work I try all user and it's not working so the problem is here.

url: (3) Port number ended with ' '
{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["Basic realm=\"security\" charset=\"UTF-8\"","Bearer realm=\"security\"","ApiKey"]}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["Basic realm=\"security\" charset=\"UTF-8\"","Bearer realm=\"security\"","ApiKey"]}},"status":401}

[stephenb (Stephen Brown)]

I had a typo missing the -u no space before the password

curl --insecure -u "user:password" https://192.168.66.11:9200

[Dreinale (Enzo Brunet)]

Ok the curl is good

curl --insecure -u "elastic:000" https://192.168.66.11:9200
{
  "name" : "epnp",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "1GLANPIBR7mInfNddGz0YQ",
  "version" : {
    "number" : "8.2.0",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "b174af62e8dd9f4ac4d25875e9381ffe2b9282c5",
    "build_date" : "2022-04-20T10:35:10.180408517Z",
    "build_snapshot" : false,
    "lucene_version" : "9.1.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

[stephenb (Stephen Brown)]

Okay so you have connectivity That is good!

Now You can also test the curl with the CA cert using this option

--cacert my-ca.crt

You can even put in -v to get all sorts of details.

Take out
--insecure

Then go back to metricbeat and start with the verification none and see that connects.

And then worry about the CA authority You may need to convert it to a PEM

[Dreinale (Enzo Brunet)]

OK i got that error

curl --cacert /etc/elasticsearch/certs/http_ca.crt -u "elastic:000" https://192.168.66.11:9200

curl: (77) error setting certificate verify locations:
  CAfile: /etc/elasticsearch/certs/http_ca.crt
  CApath: /etc/ssl/certs

I try to fix it later

[stephenb (Stephen Brown)]

It could be permission errors for the cert looks like curl can't access that file

Probably because the /etc/Elasticsearch directory is secured or you could try to set the file readable by all but the parent directory could still be non-readable

chmod 666 /etc/elasticsearch/certs/http_ca.crt

or put a copy in the metricbeat directory

You can start with the verification none until you get it sorted out.

[Dreinale (Enzo Brunet)]

Thank you for all response but i need to pause this topic for now