Filebeat Suricata Module "module suricata is configured but has no enabled filesets"

[Aaron_C_de_Bruyn]

I'm running filebeat 8.6.2.

I initially had it grabbing /var/log/remote.log and it worked fine.

Then I enabled the suricata module and set the configuration to this (excluding the output.elasticsearch section):

filebeat.inputs:
  - type: log
    paths:
      - /var/log/remote.log
    processors:
      - add_host_metadata: ~
      - add_cloud_metadata: ~
      - syslog:
          field: message
          format: auto
          ignore_missing: false
          ignore_failure: false

filebeat.modules:
  - module: system
    syslog:
      enabled: true
    auth:
      enabled: true
  - module: suricata
    eve:
      enabled: true
      var.paths:
        - "/var/log/suricata/eve.json"
      var.internal_networks:
        - private

running "filebeat -e run" ends with:

"log.level":"error","@timestamp":"2023-04-14T00:02:24.629Z","log.origin":{"file.name":"instance/beat.go","file.line":1071},"message":"Exiting: Failed to start crawler: creating module reloader failed: could not create module registry for filesets: module suricata is configured but has no enabled filesets","service.name":"filebeat","ecs.version":"1.6.0"}
Exiting: Failed to start crawler: creating module reloader failed: could not create module registry for filesets: module suricata is configured but has no enabled filesets

Digging around Google and GitHub shows various issues with running "filebeat -e setup" and pipeline issues. I had no problem running setup and Elasticsearch is populated properly. Filebeat just won't start.

It seems like a bug....because if I set the suricata module eve.enabled: false, filebeat will spit out 12 lines of output and say Exiting: module suricata is configured but has no enabled filesets.

If I set eve.enabled: true, filebeat will spit out:
Exiting: Failed to start crawler: creating module reloader failed: could not create module registry for filesets: module suricata is configured but has no enabled filesets

Additionally, I see:

{"log.level":"info","@timestamp":"2023-04-14T00:35:52.590Z","log.logger":"modules","log.origin":{"file.name":"fileset/modules.go","file.line":120},"message":"Enabled modules/filesets: system (auth), system (syslog), suricata (eve)","service.name":"filebeat","ecs.version":"1.6.0"}

...a few log lines into filebeat starting.

Thinking it had something to do with the dynamic reloading, I set filebeat.config.inputs.reload.enabled: false to no avail.

It there something broken in the module reloader related to filesets?

[stephenb (Stephen Brown)]

Hi @Aaron_C_de_Bruyn

Let me guess you enabled the suricata module.

But then you copied the suricata config into the filebeat.yml instead of just editing the config in the modules.d/suricata.yml

What happens is the one in the modules gets concatenated onto the end and then overrides what you put in filebeat.yml

Whereas you should have just edited the suricata.yml in the modules directory

That's my guess. Seen this era before. Perhaps I'm wrong.n

[Aaron_C_de_Bruyn]

Huh. I thought I was supposed to enable the module and then override the settings in filebeat.yml. I think the docs here (Suricata module | Filebeat Reference [8.7] | Elastic) aren't very clear on that.

I removed all references to suricata from filbeat.yml and updated it to this (excluding output.elasticsearch):

filebeat.inputs:
  - type: log
    paths:
      - /var/log/remote.log
    processors:
      - add_host_metadata: ~
      - add_cloud_metadata: ~
      - syslog:
          field: message
          format: auto
          ignore_missing: false
          ignore_failure: false

filebeat.modules:
  - module: system
    syslog:
      enabled: true
    auth:
      enabled: true

Then I edited suricata.yml to enable it and set the path:

# Module: suricata
# Docs: https://www.elastic.co/guide/en/beats/filebeat/8.6/filebeat-module-suricata.html

- module: suricata
  # All logs
  eve:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    var.paths: ["/var/log/suricata/eve.json"]

Same error:

{"log.level":"error","@timestamp":"2023-04-14T13:13:23.792Z","log.origin":{"file.name":"instance/beat.go","file.line":1071},"message":"Exiting: Failed to start crawler: creating module reloader failed: could not create module registry for filesets: module system is configured but has no enabled filesets","service.name":"filebeat","ecs.version":"1.6.0"}
Exiting: Failed to start crawler: creating module reloader failed: could not create module registry for filesets: module system is configured but has no enabled filesets

[Aaron_C_de_Bruyn]

Oops. Not the "same" error.

I removed the "system" module from filebeat.yml, and made similar config changes to the system.yml file and it appears to be working.

It seems like the documentation should be updated.
It gave me the impression that modules should be enabled and then configured in filebeat.yml, not that you should enable modules and then edit the module file to configure them.

[stephenb (Stephen Brown)]

@Aaron_C_de_Bruyn Glad you got it working, you are not the only on to fall into that trap..

You can open an issue against the docs if you like

From the docs on the page you referenced.

You can further refine the behavior of the suricata module by specifying variable settings in the modules.d/suricata.yml file, or overriding settings at the command line.

From the quick start page

In the module config under modules.d, change the module settings to match your environment. You must enable at least one fileset in the module. Filesets are disabled by default.