Hi, I'm adding threatintel module to filebeat , but when I restart filebeat service it failed, I get this error : Failed to start Filebeat sends log files to Logstash or directly to Elasticsearch..
Hello there!
Please paste your filebeat.yml and modules.d/threatintel.yml (using </> tag of course).
- module: threatintel
abusemalware:
enabled: true
var.input: httpjson
var.url: https://urlhaus-api.abuse.ch/v1/payloads/recent/
var.interval: 60m
I had already enabled threat intel module : sudo filebeat modules enable threatintel
Three questions:
- Is filebeat starting properly when you disable this module?
- Is this complete module config file?
- Could you please paste filebeat.yml (covering your sensitive data)?
When i disable this module filebeat is starting normaly , for the moment this all config file
in filebeat.yml it seems that it send logs to kafka , do you think may cause issue ??
Share your entire filebeat.yml.
If your filebeat already has a kafka output, you cannot configure other outputs, filebeat only supports one output.
filebeat:
config:
inputs:
enabled: true
path: /etc/filebeat/inputs.d/*.yml
modules:
enabled: true
path: /etc/filebeat/modules.d/*.yml
processors:
- add_fields:
fields:
environment: test
groups:
- auditbeat_hosts
target: inventory
- add_fields:
fields:
criticity: low
target: service
##############################################################
############################# Output ##########################################
output:
kafka:
codec.json:
pretty: false
compression: gzip
hosts:
- X.X.X.X:9092
max_message_bytes: 1000000
partition.round_robin:
reachable_only: true
required_acks: 1
topic: filebeat
And what is the error you get? You need to provide the log error that you receive when you start filebeat with the threatintel module activate, the message you shared in your first post is not enough to understand what could be happening.
filebeat.service: Service RestartSec=100ms expired, scheduling restart.
Aug 19 12:37:33 test systemd[1]: filebeat.service: Scheduled restart job, restart counter is at 5.
Aug 19 12:37:33 test systemd[1]: Stopped Filebeat sends log files to Logstash or directly to Elasticsearch..
Aug 19 12:37:33 test systemd[1]: filebeat.service: Start request repeated too quickly.
Aug 19 12:37:33 test systemd[1]: filebeat.service: Failed with result 'exit-code'.
Aug 19 12:37:33 test systemd[1]: Failed to start Filebeat sends log files to Logstash or directly to Elasticsearch..
This also does not help, you need to check the systems logs, /var/log/messages or /var/log/syslog depending on the linux distribution.
There you will find the reason why the filebeat service could not start.
2022-08-19T15:20:15.569+0200 ERROR instance/beat.go:958 Exiting: Index management requested but the Elasticsearch output is not configured/enabled
Exiting: Index management requested but the Elasticsearch output is not configured/enabled
Usually this error shows when in filebeat.yml (or other conf file) you enabled this lines:
#setup.template.settings:
#index.number_of_shards: 1
#index.codec: best_compression
#_source.enabled: false
but Elasticsearch output is disabled.
finaly , I find these lines in filebeat.reference.yml but there are disabled
Actually , I have this error
Exiting: Failed to start crawler: creating module reloader failed: fileset threatintel/abusemalware is configured but doesn't exist
This file is only a reference, not a real config file used to set things up and run filebeat.
You should add to you module conf file:
abusemalware:
enabled: false
if you don't want to use it.
the main purpose is to add threat intel module , this module use abuseurl (filesets) so i think it must be enabled
I have this error ,when I restart filebeat
[34492]: 2022-08-23T10:09:01.575+0200 INFO log/input.go:152 Configured paths: [/var/log/auth.log* /var/log/secure*]
Aug 23 10:09:01[34492]: 2022-08-23T10:09:01.575+0200 INFO log/input.go:152 Configured paths: [/var/log/messages* /var/log/syslog*]
Aug 23 10:09:01[34492]: 2022-08-23T10:09:01.575+0200 INFO beater/crawler.go:148 Stopping Crawler
Aug 23 10:09:01 [34492]: 2022-08-23T10:09:01.575+0200 INFO beater/crawler.go:158 Stopping 0 inputs
Aug 23 10:09:01 [34492]: 2022-08-23T10:09:01.575+0200 INFO beater/crawler.go:178 Crawler stopped
Aug 23 10:09:01[34492]: 2022-08-23T10:09:01.575+0200 INFO registrar/registrar.go:367 Stopping Registrar
Aug 23 10:09:01 [34492]: 2022-08-23T10:09:01.575+0200 INFO registrar/registrar.go:293 Ending Registrar
Aug 23 10:09:01[34492]: 2022-08-23T10:09:01.580+0200 INFO [monitoring] log/log.go:153 Total non-zero metrics {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":20,"time":{"ms":24}},"total":{"ticks":40,"t
Aug 23 10:09:01[34492]: 2022-08-23T10:09:01.580+0200 INFO [monitoring] log/log.go:154 Uptime: 35.592538ms
Aug 23 10:09:01 [34492]: 2022-08-23T10:09:01.580+0200 INFO [monitoring] log/log.go:131 Stopping metrics logging.
Aug 23 10:09:01[34492]: 2022-08-23T10:09:01.580+0200 INFO instance/beat.go:469 filebeat stopped.
Aug 23 10:09:01 [34492]: 2022-08-23T10:09:01.580+0200 ERROR instance/beat.go:958 Exiting: Failed to start crawler: creating module reloader failed: fileset threatintel/abuseurl is configured but doesn't exist
Aug 23 10:09:01 [34492]: Exiting: Failed to start crawler: creating module reloader failed: fileset threatintel/abuseurl is configured but doesn't exist
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.