Beats configuration: overlaping between output.elasticsearch and cloud.*?

[ffknob (Flávio Knob)]

Hi

I'm just curious about the needed settings in a Beat file when the cluster is in the Elastic cloud.

Currently my config files looks like this:

output.elasticsearch:
  hosts: ["https://....us-east-1.aws.found.io:..."]
  username: "monitor"
  password: "${MONITOR_PASSWORD}"
  compression_level: 3

setup.kibana:
  host: "....us-east-1.aws.found.io:..."
  protocol: "https"

cloud.id: "monitor:...=="
cloud.auth: "monitor:${MONITOR_PASSWORD}"

The docs state that I could use the cloud.* output to make things easier. But that output doesn't have settings like the compression_level for instance.

Could the above configuration be reduced to something like this:

output.elasticsearch:
  compression_level: 3

cloud.id: "monitor:...=="
cloud.auth: "monitor:${MONITOR_PASSWORD}"

1. Should I choose between output.elasticsearch or cloud.*?
2. Shoul I use both because I want to set the compression_level?
3. What is the setup.kibana used for? Only to set the dashboards one time for cluster?

Thank you

[jsoriano (Jaime Soriano)]

Hey @ffknob,

Take a look to this documentation page: https://www.elastic.co/guide/en/beats/filebeat/7.6/configure-cloud-id.html

In principle cloud.auth and cloud.id only overwrite the options metioned there. Rest of the options can be used the same as if cloud settings were not being used.

[ffknob (Flávio Knob)]

Thanks for the answer @jsoriano

I've just read that too fast... Here is the answer in the docs you provided:

The Cloud ID, which can be found in the Elastic Cloud web console, is used by Filebeat to resolve the Elasticsearch and Kibana URLs. This setting overwrites the output.elasticsearch.hosts and setup.kibana.host settings.

So I guess I should set cloud.* for the Beat to know to where it should send its data (output.elasticsearch.hostsl), but I could also set output.elasticsearch.compress_level for example.

Thank you

[jsoriano (Jaime Soriano)]

Yes, that's it.