Filebeats Threat Intel Module integration with Logstash

[viera120]

Hi,

I have the following self hosted setup on ELK stack 8.6.1 :

Firewall (logs) --> Filebeat --> Logstash --> Elasticsearch Cluster

I am trying to integrate FIleBeats Threat Intel Module into this setup so that IOCs in logs can be identified.

Can this be done using the existing Filebeat instance or does it require a separate dedicate instance of Filebeat?

The present filebeat.yml has output enabled for logstash:

output.logstash:
  hosts: ["192.168.1.1:5144"]

I am assuming that to integrate Threat Intel data, the threat feed would be sent directly to Elasticsearch whereas the Firewall logs would reach Elasticsearch via Logstash. This is because the logs are being enriched/filtered using logstash filters.

In Filebeat.yml, if we enable Elasticsearch output in addition to the existing Logstash output so as to send Threat intel feed to Elasticsearch, how will filebeat decide which data (logs, threat intel) goes where?

I have tried following the official documentation. However, I am not able to figure out how the Threat Intel feed can be integrated into the existing setup.

What could i possibly be missing here?

[leandrojmp (Leandro Pereira)]

Filebeat modules relies on ingest pipelines that are executed by Elasticsearch, but Filebeat only supports one output.

If your filebeat is sending data to logstash you still can use the modules, but you need to follow this documentation to configure it.

Basically you will tell logstash which ingest pipeline to use when sending the data to Elasticsearch.

Since you are also collecting Firewall logs, you will need conditionals in your logstash pipeline to tell logstash that your filters should be applied only to logs from your firewall, you can do that adding tags to the firewall input for example.

Another option that will need less work is to run a separate instance of Filebeat.

[viera120]

Thank you. Will try both options and post back!

Thanks again!