I have set up a MISP instance and have configured filebeat to query the MISP server. However, I cannot find the actual docs in any indices in logstash.
When running journalctl -fu logstash
, I see this every few minutes:
INFO [input.httpjson-cursor] v2/input.go:112 Process another repeated request. {"id": "<id>", "input_source": "https://<myserver>/events/restSearch", "input_url": "https://<myserver>/events/restSearch"}
INFO [input.httpjson-cursor] v2/request.go:210 request finished: 989 events published {"id": "<id>", "input_source": "https://<myserver>/events/restSearch", "input_url": "https://<myserver>/events/restSearch"}
I checked filebeat.yml and confirmed the output host (output.logstash), other events are being forwarded there without a problem. There are no errors in the logs.
Chaning the MISP server URL in /etc/filebeat/modules.d/threatintel.yml from https://<myserver>/events/restSearch
to https://<myserver>/
and restarting filebeat causes 0 events to be published, as opposed to 989 as shown earlier:
INFO [input.httpjson-cursor] v2/input.go:112 Process another repeated request. {"id": "<id>", "input_source": "https://<myserver>/events/restSearch", "input_url": "https://<myserver>/events/restSearch"}
INFO [input.httpjson-cursor] v2/request.go:210 request finished: 0 events published {"id": "<id>", "input_source": "https://<myserver>/events/restSearch", "input_url": "https://<myserver>/events/restSearch"}
I've tried several variations of
GET <myindex>/_search
{
"query": {
"query_string": {
"query": "(misp) OR (MISP)"
}
}
}
with no luck. Running
{
"query": {
"query_string": {
"query": "input.httpjson-cursor"
}
}
}
shows the same thing i am seeing in the commandline via journalctl, but again, not the actual events/docs. I also don't see them with any query I've tried in the Kibana Discover page and the graph never shows 989 events being ingested.
This is my filebeat.yml
- module: threatintel
abuseurl:
enabled: false
...
abusemalware:
enabled: false
...
malwarebazaar:
enabled: false
...
misp:
enabled: true
var.input: httpjson
var.url: https://<myserver>/events/restSearch
var.api_token: <mytoken>
var.ssl.verification_mode: none
var.first_interval: 300h
var.interval: 5m
otx:
enabled: false
...
anomali:
enabled: false
...
anomalithreatstream:
enabled: false
...
recordedfuture:
enabled: false
...
Is there a step that I'm missing??