# Add module threatintel to filebeat

**URL:** <https://discuss.elastic.co/t/add-module-threatintel-to-filebeat/312456>\
**Category:** Beats\
**Tags:** beats-module, filebeat\
**Created:** [August 19, 2022, 10:12am UTC](https://discuss.elastic.co/t/add-module-threatintel-to-filebeat/312456 "2022-08-19T10:12:25Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![dounia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dounia/32/109354_2.png) [@dounia](https://discuss.elastic.co/u/dounia)\
**Post date:** [August 19, 2022, 10:12am UTC](https://discuss.elastic.co/t/add-module-threatintel-to-filebeat/312456/1 "2022-08-19T10:12:25Z")

</div>

Hi, I'm adding threatintel module to filebeat , but when I restart filebeat service it failed, I get this error : Failed to start Filebeat sends log files to Logstash or directly to Elasticsearch..

---

<div class="post-metadata">

**Author:** ![cheshirecat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cheshirecat/32/109532_2.png) [@cheshirecat](https://discuss.elastic.co/u/cheshirecat)\
**Post date:** [August 19, 2022, 10:51am UTC](https://discuss.elastic.co/t/add-module-threatintel-to-filebeat/312456/2 "2022-08-19T10:51:47Z")

</div>

Hello there!  
Please paste your filebeat.yml and modules.d/threatintel.yml (using \</\> tag of course).

---

<div class="post-metadata">

**Author:** ![dounia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dounia/32/109354_2.png) [@dounia](https://discuss.elastic.co/u/dounia)\
**Post date:** [August 19, 2022, 10:54am UTC](https://discuss.elastic.co/t/add-module-threatintel-to-filebeat/312456/3 "2022-08-19T10:54:08Z")

</div>

```auto
- module: threatintel
  abusemalware:
    enabled: true
    var.input: httpjson
    var.url: https://urlhaus-api.abuse.ch/v1/payloads/recent/
    var.interval: 60m

```

---

<div class="post-metadata">

**Author:** ![dounia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dounia/32/109354_2.png) [@dounia](https://discuss.elastic.co/u/dounia)\
**Post date:** [August 19, 2022, 10:55am UTC](https://discuss.elastic.co/t/add-module-threatintel-to-filebeat/312456/4 "2022-08-19T10:55:02Z")

</div>

I had already enabled threat intel module : sudo filebeat modules enable threatintel

---

<div class="post-metadata">

**Author:** ![cheshirecat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cheshirecat/32/109532_2.png) [@cheshirecat](https://discuss.elastic.co/u/cheshirecat)\
**Post date:** [August 19, 2022, 10:57am UTC](https://discuss.elastic.co/t/add-module-threatintel-to-filebeat/312456/5 "2022-08-19T10:57:44Z")

</div>

Three questions:

1. Is filebeat starting properly when you disable this module?
2. Is this complete module config file?
3. Could you please paste filebeat.yml (covering your sensitive data)?

---

<div class="post-metadata">

**Author:** ![dounia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dounia/32/109354_2.png) [@dounia](https://discuss.elastic.co/u/dounia)\
**Post date:** [August 19, 2022, 11:00am UTC](https://discuss.elastic.co/t/add-module-threatintel-to-filebeat/312456/6 "2022-08-19T11:00:27Z")

</div>

When i disable this module filebeat is starting normaly , for the moment this all config file

---

<div class="post-metadata">

**Author:** ![dounia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dounia/32/109354_2.png) [@dounia](https://discuss.elastic.co/u/dounia)\
**Post date:** [August 19, 2022, 11:07am UTC](https://discuss.elastic.co/t/add-module-threatintel-to-filebeat/312456/7 "2022-08-19T11:07:18Z")

</div>

in filebeat.yml it seems that it send logs to kafka , do you think may cause issue ??

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [August 19, 2022, 12:05pm UTC](https://discuss.elastic.co/t/add-module-threatintel-to-filebeat/312456/8 "2022-08-19T12:05:23Z")

</div>

Share your entire `filebeat.yml`.

If your filebeat already has a `kafka` output, you cannot configure other outputs, filebeat only supports one output.

---

<div class="post-metadata">

**Author:** ![dounia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dounia/32/109354_2.png) [@dounia](https://discuss.elastic.co/u/dounia)\
**Post date:** [August 19, 2022, 12:26pm UTC](https://discuss.elastic.co/t/add-module-threatintel-to-filebeat/312456/9 "2022-08-19T12:26:29Z")

</div>

```auto
filebeat:
  config:
    inputs:
      enabled: true
      path: /etc/filebeat/inputs.d/*.yml
    modules:
      enabled: true
      path: /etc/filebeat/modules.d/*.yml
processors:
- add_fields:
    fields:
      environment: test
      groups:
      - auditbeat_hosts
    target: inventory
- add_fields:
    fields:
      criticity: low
    target: service

##############################################################
############################# Output ##########################################

output:
  kafka:
    codec.json:
      pretty: false
    compression: gzip
    hosts:
    - X.X.X.X:9092
    max_message_bytes: 1000000
    partition.round_robin:
      reachable_only: true
    required_acks: 1
    topic: filebeat

```

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [August 19, 2022, 1:00pm UTC](https://discuss.elastic.co/t/add-module-threatintel-to-filebeat/312456/10 "2022-08-19T13:00:56Z")

</div>

And what is the error you get? You need to provide the log error that you receive when you start filebeat with the threatintel module activate, the message you shared in your first post is not enough to understand what could be happening.

---

<div class="post-metadata">

**Author:** ![dounia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dounia/32/109354_2.png) [@dounia](https://discuss.elastic.co/u/dounia)\
**Post date:** [August 19, 2022, 1:11pm UTC](https://discuss.elastic.co/t/add-module-threatintel-to-filebeat/312456/11 "2022-08-19T13:11:28Z")

</div>

```auto
 filebeat.service: Service RestartSec=100ms expired, scheduling restart.
Aug 19 12:37:33 test systemd[1]: filebeat.service: Scheduled restart job, restart counter is at 5.
Aug 19 12:37:33 test systemd[1]: Stopped Filebeat sends log files to Logstash or directly to Elasticsearch..
Aug 19 12:37:33 test systemd[1]: filebeat.service: Start request repeated too quickly.
Aug 19 12:37:33 test systemd[1]: filebeat.service: Failed with result 'exit-code'.
Aug 19 12:37:33 test systemd[1]: Failed to start Filebeat sends log files to Logstash or directly to Elasticsearch..

```

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [August 19, 2022, 1:13pm UTC](https://discuss.elastic.co/t/add-module-threatintel-to-filebeat/312456/12 "2022-08-19T13:13:16Z")

</div>

This also does not help, you need to check the systems logs, `/var/log/messages` or `/var/log/syslog` depending on the linux distribution.

There you will find the reason why the filebeat service could not start.

---

<div class="post-metadata">

**Author:** ![dounia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dounia/32/109354_2.png) [@dounia](https://discuss.elastic.co/u/dounia)\
**Post date:** [August 19, 2022, 1:20pm UTC](https://discuss.elastic.co/t/add-module-threatintel-to-filebeat/312456/13 "2022-08-19T13:20:59Z")

</div>

```auto
2022-08-19T15:20:15.569+0200	ERROR	instance/beat.go:958	Exiting: Index management requested but the Elasticsearch output is not configured/enabled
Exiting: Index management requested but the Elasticsearch output is not configured/enabled

```

---

<div class="post-metadata">

**Author:** ![cheshirecat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cheshirecat/32/109532_2.png) [@cheshirecat](https://discuss.elastic.co/u/cheshirecat)\
**Post date:** [August 19, 2022, 1:43pm UTC](https://discuss.elastic.co/t/add-module-threatintel-to-filebeat/312456/14 "2022-08-19T13:43:10Z")

</div>

Usually this error shows when in filebeat.yml (or other conf file) you enabled this lines:

```auto
#setup.template.settings:
  #index.number_of_shards: 1
  #index.codec: best_compression
  #_source.enabled: false

```

but Elasticsearch output is disabled.

---

<div class="post-metadata">

**Author:** ![dounia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dounia/32/109354_2.png) [@dounia](https://discuss.elastic.co/u/dounia)\
**Post date:** [August 19, 2022, 1:52pm UTC](https://discuss.elastic.co/t/add-module-threatintel-to-filebeat/312456/15 "2022-08-19T13:52:28Z")

</div>

finaly , I find these lines in filebeat.reference.yml but there are disabled

---

<div class="post-metadata">

**Author:** ![dounia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dounia/32/109354_2.png) [@dounia](https://discuss.elastic.co/u/dounia)\
**Post date:** [August 19, 2022, 2:10pm UTC](https://discuss.elastic.co/t/add-module-threatintel-to-filebeat/312456/16 "2022-08-19T14:10:21Z")

</div>

Actually , I have this error

```auto
Exiting: Failed to start crawler: creating module reloader failed: fileset threatintel/abusemalware is configured but doesn't exist

```

---

<div class="post-metadata">

**Author:** ![cheshirecat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cheshirecat/32/109532_2.png) [@cheshirecat](https://discuss.elastic.co/u/cheshirecat)\
**Post date:** [August 19, 2022, 2:15pm UTC](https://discuss.elastic.co/t/add-module-threatintel-to-filebeat/312456/17 "2022-08-19T14:15:31Z")

</div>

> [@dounia](#):
>
> finaly , I find these lines in filebeat.reference.yml but there are disabled

This file is only a reference, not a real config file used to set things up and run filebeat.

> [@dounia](#):
>
> Actually , I have this error

You should add to you module conf file:

```auto
  abusemalware:
    enabled: false

```

if you don't want to use it.

---

<div class="post-metadata">

**Author:** ![dounia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dounia/32/109354_2.png) [@dounia](https://discuss.elastic.co/u/dounia)\
**Post date:** [August 19, 2022, 2:19pm UTC](https://discuss.elastic.co/t/add-module-threatintel-to-filebeat/312456/18 "2022-08-19T14:19:44Z")

</div>

the main purpose is to add threat intel module , this module use abuseurl (filesets) so i think it must be enabled

---

<div class="post-metadata">

**Author:** ![dounia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dounia/32/109354_2.png) [@dounia](https://discuss.elastic.co/u/dounia)\
**Post date:** [August 23, 2022, 8:18am UTC](https://discuss.elastic.co/t/add-module-threatintel-to-filebeat/312456/19 "2022-08-23T08:18:46Z")

</div>

I have this error ,when I restart filebeat

```auto
[34492]: 2022-08-23T10:09:01.575+0200 INFO log/input.go:152 Configured paths: [/var/log/auth.log* /var/log/secure*]
Aug 23 10:09:01[34492]: 2022-08-23T10:09:01.575+0200 INFO log/input.go:152 Configured paths: [/var/log/messages* /var/log/syslog*]
Aug 23 10:09:01[34492]: 2022-08-23T10:09:01.575+0200 INFO beater/crawler.go:148 Stopping Crawler
Aug 23 10:09:01 [34492]: 2022-08-23T10:09:01.575+0200 INFO beater/crawler.go:158 Stopping 0 inputs
Aug 23 10:09:01 [34492]: 2022-08-23T10:09:01.575+0200 INFO beater/crawler.go:178 Crawler stopped
Aug 23 10:09:01[34492]: 2022-08-23T10:09:01.575+0200 INFO registrar/registrar.go:367 Stopping Registrar
Aug 23 10:09:01 [34492]: 2022-08-23T10:09:01.575+0200 INFO registrar/registrar.go:293 Ending Registrar
Aug 23 10:09:01[34492]: 2022-08-23T10:09:01.580+0200 INFO [monitoring] log/log.go:153 Total non-zero metrics {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":20,"time":{"ms":24}},"total":{"ticks":40,"t
Aug 23 10:09:01[34492]: 2022-08-23T10:09:01.580+0200 INFO [monitoring] log/log.go:154 Uptime: 35.592538ms
Aug 23 10:09:01 [34492]: 2022-08-23T10:09:01.580+0200 INFO [monitoring] log/log.go:131 Stopping metrics logging.
Aug 23 10:09:01[34492]: 2022-08-23T10:09:01.580+0200 INFO instance/beat.go:469 filebeat stopped.
Aug 23 10:09:01 [34492]: 2022-08-23T10:09:01.580+0200 ERROR instance/beat.go:958 Exiting: Failed to start crawler: creating module reloader failed: fileset threatintel/abuseurl is configured but doesn't exist
Aug 23 10:09:01 [34492]: Exiting: Failed to start crawler: creating module reloader failed: fileset threatintel/abuseurl is configured but doesn't exist

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 20, 2022, 8:19am UTC](https://discuss.elastic.co/t/add-module-threatintel-to-filebeat/312456/20 "2022-09-20T08:19:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.