Exiting: Failed to start crawler: creating module reloader failed:

Elastic Stack Beats
1

using: https://www.elastic.co/guide/en/elastic-stack/current/installing-elastic-stack.html

Versions on ubuntu 22.04.

/usr/share/logstash/bin/logstash -V
Using bundled JDK: /usr/share/logstash/jdk
logstash 8.5.3

 /usr/bin/filebeat version
filebeat version 8.5.3 (amd64), libbeat 8.5.3 [6d03209df870c63ef9d59d609268c11dfdc835dd built 2022-12-04 04:51:48 +0000 UTC]

/usr/share/elasticsearch/bin/elasticsearch -V
Version: 8.5.3, Build: deb/4ed5ee9afac63de92ec98f404ccbed7d3ba9584e/2022-12-05T18:22:22.226119656Z, JVM: 19.0.1

I am trying to send logs to localhost 9200. Once i can gather local logs, I can work on remote logs. In theory.

My current problem from /var/log/syslog
Dec 19 20:56:22 ub2204elk filebeat[1432]: {"log.level":"error","@timestamp":"2022-12-19T20:56:22.075Z","log.origin":{"file.name":"instance/beat.go","file.line":1057},"message":"Exiting: Failed to start crawler: creating module reloader failed: could not create module registry for filesets: module elasticsearch is configured but has no enabled filesets","service.name":"filebeat","ecs.version":"1.6.0"}

grep -v "#" filebeat.yml |uniq

filebeat.inputs:
- type: filestream
  id: my-filestream-id
  enabled: false
  paths:
    - /var/log/*.log

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 1

setup.kibana:
  host: "localhost:5601"

output.elasticsearch:
  hosts: ["localhost:9200"]
  username: "elastic"
  password: "SCRAMBLE"

processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~

cat modules.d/system.yml

# Module: system
# Docs: https://www.elastic.co/guide/en/beats/filebeat/8.5/filebeat-module-system.html
# https://logit.io/sources/configure/filebeat-system/
- module: system
  # Syslog
  syslog:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    # var.paths: "/var/log/syslog"

  # Authorization logs
  auth:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    # var.paths: "/var/log/auth.log"

#I commented the paths. Says the OS will use the defaults.
2

It looks like you have the elasticsearch filebeat module enabled (modules.d/elasticsearch.yml instead of modules.d/elasticsearch.yml.disabled) but don't have any of the filesets in the elasticsearch module enabled.

3

I don't seem to have a .disabled.

root@ub2204elk:/etc/filebeat# find / -type f -name elasticsearch.yml.disabled 
root@ub2204elk:/etc/filebeat# find / -type f -name elasticsearch.yml
/etc/elasticsearch/elasticsearch.yml
/etc/filebeat/modules.d/elasticsearch.yml

I also noticed this, I don't know if it is related (or how it happened)

which filebeat
**/usr/bin/filebeat**

systemctl status filebeat
● filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch.
     Loaded: loaded (/lib/systemd/system/filebeat.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2022-12-19 21:55:08 UTC; 2s ago
       Docs: https://www.elastic.co/beats/filebeat
   Main PID: 7530 (filebeat)
      Tasks: 5 (limit: 4575)
     Memory: 30.6M
        CPU: 92ms
     CGroup: /system.slice/filebeat.service
             └─7530 **/usr/share/filebeat/bin/filebeat** --environment systemd -c /etc/filebeat/filebeat.yml --path.home /usr/share/filebeat --path>

both exist

root@ub2204elk:/etc/filebeat# ls -l /usr/share/filebeat/bin/filebeat
-rwxr-xr-x 1 root root 129248384 Dec  4 05:00 /usr/share/filebeat/bin/filebeat
root@ub2204elk:/etc/filebeat# ls -l /usr/bin/filebeat
-rwxr-xr-x 1 root root 335 Dec  4 05:00 /usr/bin/filebeat
4 
cat /usr/bin/filebeat
#!/usr/bin/env bash

# Script to run Filebeat in foreground with the same path settings that
# the init script / systemd unit file would do.
umask 0027
exec /usr/share/filebeat/bin/filebeat \
  --path.home /usr/share/filebeat \
  --path.config /etc/filebeat \
  --path.data /var/lib/filebeat \
  --path.logs /var/log/filebeat \
  "$@"

That isn't so dramatic. Back to examining the original error message. And why it won't load modules.

Dec 19 20:56:22 ub2204elk filebeat[1432]: {"log.level":"error","@timestamp":"2022-12-19T20:56:22.075Z","log.origin":{"file.name":"instance/beat.go","file.line":1057},"message":"Exiting: Failed to start crawler: creating module reloader failed: could not create module registry for filesets: module elasticsearch is configured but has no enabled filesets","service.name":"filebeat","ecs.version":"1.6.0"}
5

Right. The fact that you have a file called /etc/filebeat/modules.d/elasticsearch.yml, means that the elasticsearch module is enabled. If you rename the file to /etc/filebeat/modules.d/elasticsearch.yml.disabled that will disable the elasticsearch module.

The error is because you have the elasticsearch module enabled, but haven't enabled any filesets in the module.

6

fixed. I am now geting localhost's rsyslogs into mykibana 5601. I am still struggling with remote logs being collected.

My question now is.., Does remote rsyslog send direction into filebeats on the local, or does filebeats need installed on the remote server as well?

The port for logstash is open from the remote server and I can "telnet elkIP filebeatport" and connect fine.

7

That is more of an architectural decision. For filebeat it is more common to install filebeat on each host. You can setup filebeat to listen for incoming syslog messages (Syslog input | Filebeat Reference [8.5] | Elastic) but it is more common to use Logstash for that use case.

8

I don't suppose you have a linky to installing and configuring filebeats on the remote hosts (servers to be monitored) I'm kinda confused right now.

With tcpdump running on both elk server and remote host I can see the exchange of data on the specified port (of 10514) during a log write to the remote host. It looks like good communication, but, nothing gets written to kibana | Observability | Logs | Stream. Is this the correct place? Is it being filtered out?)

I put

# FOR remote rsyslog
filebeat.inputs:
- type: syslog
  format: auto
  protocol.unix:
    host: "localhost:10514"

into /etc/filebeat/filebeat.yml on the elk server.

on the remote guy in /etc/rsyslog.d/70-output.conf
*.* @@192.168.1.205:10514;json-template (
.205 is the elk server)

filebeat is not installed on remote. Is that the way I need to go?

9 
protocol.unix

is unusual for syslog from remote systems. Normally I would expect protocol.udp. What did the tcpdump show as the protocol?

10

My solution was to send remote rsyslog to elk's rsyslog, and then file beats picked up the logs locally. I would like to see these logs encrypted at some point, but I'm satisfied with this solution for now.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.