We are using ELK 7.11.1 version.
We have ingested threat intelligence feeds from MISP server and stored in Elasticsearch through Filebeat.(Using Module MISP) and we want to map and match with Zscalerlog.
Based on the above condition, we have created indicator match rule.
The following way we had tested.
- I took one malicious URL from MISP server attribute lists and get it browsed via browser (http://ww.gzcfr5axf6.com/) , In the filebeat-*-MISP indices : misp.threat_indicator.attack_pattern_kql and from kibana discover it is shown as misp.threat_indicator.attack_pattern_kql: source.domain: "ww.gzcfr5axf6.com" OR destination.domain: "ww.gzcfr5axf6.com".
- In the zscalerlog indice , url.destination shown as url.destination: http://ww.gzcfr5axf6.com/ where by this results will be shown once we accessed to it via web browser.
we have many times tested, but the results didn't match with filebeat-*-MISP and also we got alerts but incorrect results, it is only show first 100 Zscalerlog record without match and map with Filebeat-*-MISP, the rule run every 5 minutes scheduling .
Please see our rule:-
Please see our result:-
- Filebeat-*- MISP don't have duplicate records.
- Filebeat-*- MISP current total records is 600k
- ZscalerLog is real-time logs.
- ZscalerLog, each 10 minutes once will receive 3k records.
Can you help me?