I am trying to follow this tutorial here to build a dashboard on some public data sets.
However, whenever I tried to run ./filebeat, the following error appeared:
2021-03-31T14:19:44.685+0800 ERROR [modules] fileset/modules.go:131 Not loading modules. Module directory not found: /usr/share/filebeat/bin/module
And I have found this post here
I see the same info popping up in the terminal, meaning that the path are not configured correctly.
2020-06-08T17:49:34.832-0500 INFO instance/beat.go:622 Home path: [/usr/share/filebeat/bin] Config path: [/etc/filebeat] Data path: [/usr/share/filebeat/bin/data] Logs path: [/usr/share/filebeat/bin/logs]
I tried modifying /etc/filebeat/filebeat.yml as stated in here, but it doesn't work.
Then I noted that there's some customized configuration in this example data set(you need to move a custom .yml (ncedc-earthquakes-filebeat.yml) to the /usr/share/filebeat/ directory). Then I tried to add the above configuration to this
By default filebeat uses the current directory as home path, it seems that you are running filebeat as ./filebeat from the /usr/share/filebeat/bin directory, and then it considers this directory its home path.
When installing using RPM or Debian packages a wrapper is installed in one of the default paths (in /usr/bin/filebeat). This wrapper includes flags to setup the proper paths:
# Script to run Filebeat in foreground with the same path settings that
# the init script / systemd unit file would do.
exec /usr/share/filebeat/bin/filebeat \
--path.home /usr/share/filebeat \
--path.config /etc/filebeat \
--path.data /var/lib/filebeat \
--path.logs /var/log/filebeat \
In summary, try running filebeat without ./, so the wrapper is used. Alternatively, run filebeat with custom paths.
Also take into account that the tutorial you are following is created for Filebeat 6.0, you may find some differences with more recent versions.
I actually realised I forgot to run filebeat at the same time as logstash which is why there's no output. (yes that's stupid)
Btw, a brute-force method could be building a soft link between /usr/share/filebeat/bin and /usr/share/filebeat/bin/module as well. (by doing ln -s)
I tried it before and it silents the error(though, back then I haven't realised I didn't turn on Logstash so there's still no output, but I think it's worth a shot)