Metricbeats using port 9243 instead of 443

[alphabet5]

Using elastic cloud I am trying to connect metricbeats with the provided cloud.id/auth.

I'm seeing this in the logs

{"log.level":"info","@timestamp":"2022-03-25T05:56:18.524Z","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":141},"message":"Attempting to reconnect to backoff(elasticsearch(https://{elastic_url}.us-east-2.aws.elastic-cloud.com:9243)) with 250 reconnect attempt(s)","service.name":"metricbeat","ecs.version":"1.6.0"}

The docs here state that it should be using 443 by default.

The base64 encoded cloud.id found in the Elasticsearch Service web console does not explicitly specify a port. This means that Metricbeat will default to using port 443 when using cloud.id, not the commonly configured cloud endpoint port 9243.

Any idea why 9243 would be showing up in the logs if 443 should be used by default?

Using docker.elastic.co/beats/metricbeat:8.1.1 with a pretty simple metricbeat.yml

cloud:
  id: ${CLOUDID}
  auth: ${CLOUDAUTH}
metricbeat.modules:
- module: vsphere
  enabled: true
  metricsets: ["datastore", "host", "virtualmachine"]
  period: 10s
  hosts: ["https://1.2.3.4/sdk"]
  username: ${USERNAME}
  password: ${PASSWORD}
  # If insecure is true, don't verify the server's certificate chain
  insecure: true
  # Get custom fields when using virtualmachine metric set. Default false.
  # get_custom_fields: false

[angelo]

It could be due to using an 'older' cloud.id that still has port 9243 as part of it. You should base64 decode the part of the cloud.id to confirm the URL does not have a port specified.

If it does, you could regenerate a new version of the cloud.id that either excludes the port (9243) or explicitly specifies 443.

[alphabet5]

That was exactly the issue. Thanks for the help! I didn't see the :9243 embedded in there before.

[alphabet5]

Do you have information on how to generate a new cloud id? Does that require a new deployment? Not finding anything in the documentation.

[angelo]

It does not require a new deployment. Take the decoded portion and edit it as you want (replace 9243 with 443 or completely remove :9243) and just base64 encode it again. The format of the Cloud ID is the deployment name, a colon and then the encoded portion is the Cloud Provider/Region domain (with optional port number), Elasticsearch Cluster ID and Kibana ID separated by '$' characters.

Example - replace vars with your values and if you want the port or not:

echo -n '{cloud_domain}{:optional_port}${es_id}${kibana_id}' | base64

• Make sure to use the '-n' option for the echo command so as to not encode any CR/LF chars as part of new cloud.id you are generating.

Once you have that, prepend the same deployment name from your current cloud.id with a colon to this new version and use this in your configuration.