Metricbeat 6.0.0-beta1 config reload issue

[randude]

Hi,
I enabled this feature and tried several things.
I changed one of the modules from false to true and then tried to add a new module altogether.
In both cases, nothing was written to log to confirm that the reload happened and no new data was written to my output file.

Here's my conf file:

metricbeat.config.modules:

  path: /etc/metricbeat/metricbeat.yml
  reload.period: 5s


  reload.enabled: true


metricbeat.max_start_delay: 10s


metricbeat.modules:
- module: system
  metricsets: ["process"]
  processors:
   - include_fields:
      fields: ["system.process.name", "system.process.cpu.total.pct", "system.process.cmdline", "system.process.memory.size", "system.process.fd.open" ]
   - drop_event.when.not.regexp:
      system.process.name: 'sshd'
  enabled: true
  period: 30s
  processes: ['.*']
  
- module: system
  metricsets: ["filesystem"]
  enabled: true
  period: 30s
  processes: ['.*']
  processors:
   - include_fields:
      fields: ["system.filesystem.free", "system.filesystem.total", "system.filesystem.used.pct", "system.filesystem.device_name" ]
      
- module: system
  metricsets: ["memory"]
  enabled: true
  period: 30s
  processes: ['.*']
  processors:
   - include_fields:
      fields: ["system.memory.actual.free", "system.memory.total" ]
        
- module: system
  metricsets: ["network"]
  enabled: true
  period: 30s
  processes: ['.*']
  processors:
   - include_fields:
      fields: ["system.network.in.bytes", "system.network.out.bytes" ]
      
- module: system
  metricsets: ["cpu"]
  enabled: true
  period: 30s
  processes: ['.*']
  processors:
   - include_fields:
      fields: ["system.cpu.cores", "system.cpu.idle.pct" ]
  
  cpu.metrics:  ["percentages"]  # The other available options are normalized_percentages and ticks.
  core.metrics: ["percentages"]  # The other available option is ticks.
  
- module: mongodb
  metricsets:  ["dbstats", "status"]
  enabled: false
  period: 30s
  hosts: ["localhost:27017"]
  processors:
   - drop_fields:
      fields: ['beat']
        
logging.to_syslog: false
logging.metrics.enabled: true
logging.metrics.period: 1800
logging.level: debug
logging.to_files: true
logging.json: false

logging.files:
  path: /var/log/metricbeat
  name: metricbeat.log
  rotateeverybytes: 10485760
  keepfiles: 7
  
output.file:
  path: "/var//log/metricbeat/"
  filename: metricbeat.txt
  #rotate_every_kb: 10000
  number_of_files: 2

[exekias (Carlos Pérez Aradros)]

Hi @randude,

From your config I see:

metricbeat.config.modules:
  path: /etc/metricbeat/metricbeat.yml
  reload.period: 5s
  reload.enabled: true

But metricbeat expects metricbeat.config.modules.path to be a pattern of files to load, check https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-configuration-reloading.html for more details. I think you want to create a new folder and put your modules on separate files on it

Best regards

[randude]

Hi, thanks for the reply @exekias
I enabled the log level to be debug and i saw that it did find my pattern. The path that i provided is a legal regex after all.
However, it said that it didnt detect any change.
So just to clarify, does it load only new files or can it actually detect changes in existing modules i.e disabling/adding new processors?

[exekias (Carlos Pérez Aradros)]

It can detect changes, after loading a valid configuration

The issue in this case is that when it tries to load metricbeat.ymlit doesn't find a module, but the whole settings for metricbeat, which is not a valid config for a module.

[randude]

I see! That explains it haha.
Thanks

You should add that to the official doc.
"Each file found by the Glob must contain only a list of one or more module definitions. "

[randude]

So I got it to work. I set the path to "path: /etc/metricbeat/modules.yml" and had to remove the "metricbeat.modules:" headline.
Thanks!