I'm looking for a SIEM solution for our company. At the moment I'm trying to do it with ELK.
My goal is:
I would like to create fancy dashboards, alerts with events correlated from Threat Intel.
I would like to update this dashboard and alarm every time a new IOC match was found (retrospective analysis).
I tried to find an official elastic security/SIEM supported way to do it. I thought it is basic SIEM feature so there has to be a solution. But I couldn't find any answer.
My initial idea was to import threat intel to Elasticsearch and correlate events from the output create a special "dynamic" view and use it as a dashboard, and alert source for SIEM detection rules (SQL way of thinking).
Threat feed to Elastic SIEM:
It seems Elastic SIEM can't handle threat intel natively, so I have to install another server with MISP and export feeds from there to ELK SIEM.
I have found ways to do that so it seems doable, I can delete old IOC-s I'm not interested in...
Issues: I isn't an efficient way, and MISP isn't supported and maintained by Elastic.
Threat feed into Elastic: partially checked.
Use detection rules to trigger alerts:
Before version 7.10 it was impossible from SIEM (watcher is not a detection rule...).
Starting from 7.10 there is a slight hope. There is a new feature called "Indicator Match".
As far as I understand it can be used to check if there is a match between IOC and my Log Files, sounds great.
The issue is: "the resulting detection alert does not contain explicit information about which event field(s) match which indicator field(s)".
What I can use is to " include a reference to the field(s) to be matched in the rule Name and rule Description, and ensure that the Timeline template associated with the rule includes pre-defined column(s) for these fields."
So the result will be: receive an alert, have a slight idea where to start digging.
Unfortunately I can't find any way to create events form detection rules and highlight a single log entry.
As far as I see correlation with Elastic is a nightmare.
I can't create a KQL to correlate two fields from different indexes.
I can't create Query DSL to correlate two fields from different indexes.
I can't create an SQL like query which include join and sub query-s to correlate two fields from different indexes.
If I receive an alert I have to export the affected logs and manually correlate them with IOC-s by custom-made scripts or feed it to an SQL DB... It isn't an efficient way...
Alerts from SIEM: there is a half-baked solution, it requires a lot of manual work in order to correlate.
Create a dashboard from the correlated events (and it has to be dynamically updated with new IOC-s...):
The only way I found is enrich the data before indexing unfortunately by doing that I lose the retrospective analysis. Ouch.
It can be done by Logstash or by Enrich processor.
The logstash way: a lot of people try to use memcached in order to enrich data and index high volume of events.
The issues with this: I have to import IOC-s to Elastic, then I import IOC-s to memcached with a different tool and consume more resources, memcached itself isn't supported by elastic (there is a plugin but they don't support the product itself). It doubles the maintenance costs.
It seems to me that Elastic is focused on Beats... not on Logstash, I read Logstash isn't efficient and I should minimize the usage of it and focus on the new way: Beats.
I thought I can use Beats to enrich data. Unfortunately I can't find any way to do that.
For example I would like to use Filebeat with netflow module and enrich the data with IOC-s form MISP by memcached?
The solution I found is to send beats output to Logstash and re parse the beats output and enrich it.
It seems like there is a contradiction. Try to use Beats it is light and efficient ohh wait use it with Logstas to enrich...
The next idea was to use the enrich processor.
Read "The approach of keeping threat intel in ES and checking against it for every ingested log entry is quite heavy". So it seems to be another dead end.
I spent 1 month to figure out how to deal with Threat Intel in Elastic SIEM.
Before I give up.
Is there an efficient and easy way to use Threat Intel with Elastic SIEM built in tools?
If there is please send me a link.