Metricbeat setup: one-time?

Elastic Stack Beats
1

Is metricbeat setup meant to be run once per cluster?

The documentation (Metricbeat quick start: installation and configuration | Metricbeat Reference [8.6] | Elastic) does not say.

It is implied that this command is meant to be run more often, e.g. once per host, as the documentation contains instructions for creating a setup role (Grant privileges and roles needed for setup | Metricbeat Reference [8.6] | Elastic). That would probably be overkill for a one-time action.

However, a DigitalOcean guide (How To Gather Infrastructure Metrics with Metricbeat on Ubuntu 18.04 | DigitalOcean) instructs users to run metricbeat setup on the first host only (two hosts are set up in the tutorial).

2

@NominaSumpta

Great Question setup loads templates, mappings, ingest pipelines, dashboards etc...

metricbeat setup -e

Needs to be run once each time you change the version or change the configuration, like add module, add inputs.etc

Say you install metricbeat on a host and enable the system module... You run metricbeat setup -e ...

You can then install metricbeat on as many servers as you like without running setup because all the config is already loaded.

Then you decide to add the MySQL module you need to run setup again once, and then update the configurations / modules or install on your 10 MySQL hosts.

Then you upgrade a version you need to run setup once again.

Basically once per configuration change independent of how many hosts it is installed on.

Hope that helps.

1 Like
3

Hi Stephen,

Then you decide to add the MySQL module you need to run setup again once

Just to check: is this needed for every module? Having run setup on a host with no modules enabled, dashboards for all modules were created in Kibana already.

Then you upgrade a version you need to run setup once again.

Is this the case for minor or major versions?

Thank you for your reply!

4

What version you on?

System module us enabled by default.

Hmm that should not be the expected behavior but yes run each time you add a module ... There are other assets loaded..

Say first time you have 3 modula...run setup once... Then you add 2 more modula tun it once.

Both major and minor, after all there could be bug fixes.

1 Like
5

I assume the same goes for other Elastic Stack "extras"? Every time we bump the version of Filebeat, Metricbeat, Elastic APM, etc, we should check if there is something like a "setup -e" option that can be run once (per cluster)?

How about major components like Logstash and Kibana? Do they even have a "setup -e" option?

I need to check if our CI/CD pipelines do such a thing.

6

Beats have setup, Logstash Kibana, Elasticsearch do not, legacy APM server did I think new integration does not.

You did not answer the version.

No they do not.

1 Like
7

You did not answer the version.

That might be because I did not answer at all :slight_smile:

What version you on?

7.17.9

metricbeat setup -e

Needs to be run once each time you change the version or change the configuration, like add module, add inputs.etc

My $0.02:

In my case, Metricbeat runs on ~200 machines.

Unprivileged users have access to these. So an Elasticsearch user with the setup role can't be used.

Therefore, when a machine starts using a module that is not used by other machines, setup must be run on a dedicated machine that unprivileged users do not have access to.

This is inconvenient.

8

Yes that would be correct

Interesting. I have other users that find this convenient because they know exactly where all configuration and set up is from.

They have a setup box that they use to run the setup via CI/CD / Automation tool of choice.
That way setup / configuration is strictly controlled.

Elastic Agent flips the model because it is centralized management via Kibana and with centralized API in progress.

In other words, the configuration and setup not driven from the source systems

9

I could indeed install Metricbeat on a dedicated machine, and run setup on it when needed, but that requires me to know when configurations have changed, which is not trivial with at least the configuration management system that I use (Ansible).

I guess I should use Elastic Agent to prevent this.

Thank you for your helpful replies so far!

10

Hi @stephenb
If I install without doing setup, how do we know if the metricbeat we just installed retrieves system data if we use the system module?

11

I am sure that if you use the system module with Filebeat, then Filebeat will retrieve system data.

But the more important question is: Will you be able to see/use the data? That is where I think the setup becomes important.

12

Because the system module for metricbeat is enabled by default

Perhaps read this

If you want to run setup every time you can that is up to you, just know it is not necessary.

13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.