Can't disable metricbeat metrics module

[fozboz (Andy Foster)]

It seems that I'm unable to disable to metrics module in metricbeat. My metricbeat.yml is thusly:

output.logstash:
  hosts: ["logstash1:5044","logstash2:5044"]
  loadbalance: true

metricbeat.modules:
- module: apache
  enabled: false
  period: 60s
  hosts: ["http://127.0.0.1"]
  metricsets: ["status"]
  server_status_path: server-status
  username:
  password:
- module: haproxy
  enabled: false
  period: 60s
  hosts: ["tcp://127.0.0.1:14567"]
  metricsets: ["info","stat"]
- module: mysql
  enabled: false
  period: 60s
  hosts: ["tcp(127.0.0.1:3306)/"]
  metricsets: ["status"]
  username:
  password:
- module: mongodb
  enabled: false
  period: 60s
  hosts: ["127.0.0.1:40001"]
  metricsets: ["dbstats","status"]
  username:
  password:
- module: nginx
  enabled: false
  period: 60s
  hosts: ["http://127.0.0.1/"]
  metricsets: ["stubstatus"]
  server_status_path: server-status
- module: php_fpm
  enabled: false
  period: 60s
  hosts: ["127.0.0.1:8080"]
  metricsets: ["pool"]
  status_path: /status
- module: postgresql
  enabled: false
  period: 60s
  hosts: ["postgres://localhost:5432"]
  metricsets: ["database","bgwriter","activity"]
  username:
  password:
- module: system
  enabled: false
  period: 10s
  metricsets: ["cpu","load","core","diskio","filesystem","fsstat","memory","network","process","socket"]
  processes: [""]
- module: zookeeper
  enabled: false
  period: 60s
  metricsets: ["mntr"]
  hosts: ["localhost:2181"]

So everything is disabled, but I'm still getting metrics appearing in ElasticSearch. Moreover, metricbeat starts by logging this:

2018-05-28T15:06:46.879-0400    INFO    instance/beat.go:475    Beat UUID: 9012999a-0b52-4940-8c11-db6d2047cd85
2018-05-28T15:06:46.879-0400    INFO    instance/beat.go:213    Setup Beat: metricbeat; Version: 6.2.4
2018-05-28T15:06:46.880-0400    INFO    pipeline/module.go:76   Beat name: metrics
2018-05-28T15:06:46.881-0400    WARN    [cfgwarn]       socket/socket.go:49     BETA: The system collector metricset is beta
2018-05-28T15:06:46.900-0400    INFO    [monitoring]    log/log.go:97   Starting metrics logging every 30s
2018-05-28T15:06:46.901-0400    INFO    instance/beat.go:301    metricbeat start running.

I'm confused.

[ruflin]

Can you share how you start Metricbeat and which version you are using?

Is the above your full config file or an excerpt?

[fozboz (Andy Foster)]

Starting with systemctl start metricbeat.
Version 6.2.4 installed from the Debian repo.
That is the full config file.

[fozboz (Andy Foster)]

Update: this only occurs on 14.04.

16.04 nodes exit as expected with

2018-06-05T14:05:18.783-0400    INFO    instance/beat.go:468    Home path: [/usr/share/metricbeat] Config path: [/etc/metricbeat] Data path: [/var/lib/metricbeat] Logs path: [/var/log/metricbeat]
2018-06-05T14:05:18.783-0400    INFO    instance/beat.go:475    Beat UUID: 813a4cf9-6631-4d9b-a233-8f62a2706738
2018-06-05T14:05:18.783-0400    INFO    instance/beat.go:213    Setup Beat: metricbeat; Version: 6.2.4
2018-06-05T14:05:18.784-0400    INFO    pipeline/module.go:76   Beat name: metrics
2018-06-05T14:05:18.785-0400    ERROR   instance/beat.go:667    Exiting: all modules are disabled

[ruflin]

Can you try to run metricbeat on 14.04 manually and see if you get the same behaviour. I'm wondering if a path to a config file is wrong.

[fozboz (Andy Foster)]

Sorry, I said previously we were using systemctl but with this being 14.04, we're using the init scripts that come with the metricbeat package. I also told you we were issuing the start command but we were in fact using restart after changing config.

I can get the expected result by stopping and then starting metricbeat. So...

service metricbeat restart

results in the correct Exiting: all modules are disabled message, but the thread continues to run and metrics continue to be sent to ElasticSearch.

Running service metricbeat stop and then service metricbeat start gives me the expected outcome.

[ruflin]

For my understanding: Start / Stop works as expected. But when you run restart the old metricbeat instance does not shut down and the new one that is tried to start returns the error?

[fozboz (Andy Foster)]

Yes exactly. Just tested on a clean 1404 system.

[ruflin]

Sorry for the long delay on this on. I tried to reproduce this with a fresh ubuntu:14.04 container and installed metricbeat. It seems start / stop and restart had the same effect.