I have installed ELK stack on FreeBSD using pkg manager.
Kibana reports my license as BASIC.
I don't appear to have the suricata module.
Can someone please confirm which license i have?
You Are Basic License Suricata is part of the Basic license.
I suspect you are looking for Suricata under the ingest manager? 7.9 is the very first release with the new agent and ingest manager. It Appears Suricata logs did not make this first release. I suspect it will be supported in an upcoming release.
In the mean time you can ingest Suricata logs with the following method.
To ingest Suricata you need install Filebeat with the Suricata module.
Click on the Kibana Home button on top right, Add Security Events , click on Suricata and follow the instructions.
https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-suricata.html
Thank you for confirming i am on Basic license @stephenb
It appears I am on 7.7.1 which seems to be the latest for FreeBSD.
I have installed beats7 package but the suricata module doesn't appear to be present.
From what i can see and i am an ELK newbie, this module only creates kibana dashboards, is that right?
I have Suricata logs being sent to Redis which is being sent to logstash which is being sent to elasticsearch successfully. I think my next challenge is to implement a template in between logstash and elasticsearch using this as a guide
https://github.com/gregwjacobs/ElasticSuricata
Would you agree this would be a good next step?
Hi @bn1980
Lets see, lots to parse there... 
I am not and expert in any form on FreeBSD so someone else may need to chime in.
1st from what it looks like by the list of modules that is the OSS distribution of filebeat not the Default / Basic distribute of filebeat that is why the Suricata module is missing (a couple other are missing as well)
2nd Modules do more than just set up visualization Modules configure / contol a number of aspects.
- Where / how to collect the telemetry
- Mapping (and specifically mapping to Elastic Common Schema we will come back to that which will allow automatic use of the SIEM app)
- Ingest Pipeline : The parsing and mapping of the data on ingest
- Visualizations
- And Any other Capabilities like Anomaly Detection jobs.
3rd Now to the your approach.
I can not really comment on the the repo you posted, other than it looks like it has not been updated in 3 years.
In general if you want to take advantage of the SIEM app and existing visualizations I would map the fields to the Elastic Common Schema (ECS) and then you be able to take advantage of the SIEM apps, Anomaly Jobs etc and perhaps the dashboards... which I think is perhaps a better long term approach.
There are 2 meta approach 1 try to ingest the data into the filebeat Index or create your own index and mappings. You can add your own index to the SIEM settings so that is a fine approach.
I would get a copy of the Basic filebeat distributed and run setup with the Suricata module. If you can not find the Basic FreeBSD just load it on another OS and point it to your cluster and run setup it only needs to be run once. OR if you are putting this in a non filebeat index you will just need to create your own mapping and parsing logic. (you wont get the dashboards / viz based on filebeat for free you will need to re-create with your own index).
Then I would follow these instruction about mapping the fields to ECS.
https://www.elastic.co/guide/en/security/current/siem-field-reference.html
Not every field needs to be mapped
Here are the list of Suricata specific fields
https://www.elastic.co/guide/en/beats/filebeat/7.7/exported-fields-suricata.html
I think this approach (mapping to ECS etc) will perhaps put you on a longer term path.
Hope that helps a bit.
1 Like
Also if you have Security Analytics questions I would post the in that category, with this subject title and category here I don't think you'll get many responses
Thanks again @stephenb . It's been 24d and i still haven't got this working (but im not a developer). I tried installing filebeat on a different OS and then running the suricata module setup but that didn't work. (couldn't communicate with kibana properly). I am still exploring this.
But I'd like to create my own mapping, I'm just not sure how to do this? Would you be able to provide a some dumbed down guidance for me please?
At the moment i have Redis > Logstash > Elasticsearch fine, i THINK this is my existing mapping?
Existing Mapping
"mappings": {
"_doc": {
"dynamic_templates": [
{
"message_field": {
"path_match": "message",
"match_mapping_type": "string",
"mapping": {
"norms": false,
"type": "text"
}
}
},
{
"string_fields": {
"match": "*",
"match_mapping_type": "string",
"mapping": {
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
},
"norms": false,
"type": "text"
}
}
}
],
"properties": {
"@timestamp": {
"type": "date"
},
"@version": {
"type": "keyword"
},
"alert": {
"properties": {
"action": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"category": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"gid": {
"type": "long"
},
"rev": {
"type": "long"
},
"severity": {
"type": "long"
},
"signature": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"signature_id": {
"type": "long"
}
}
},
"app_proto": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"dest_geoip": {
"properties": {
"city_name": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"continent_code": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"country_code2": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"country_code3": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"country_name": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"dma_code": {
"type": "long"
},
"ip": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"latitude": {
"type": "float"
},
"location": {
"properties": {
"lat": {
"type": "float"
},
"lon": {
"type": "float"
}
}
},
"longitude": {
"type": "float"
},
"postal_code": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"region_code": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"region_name": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"timezone": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"dest_ip": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"dest_port": {
"type": "long"
},
"event_type": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"flow_id": {
"type": "long"
},
"geoip": {
"dynamic": "true",
"properties": {
"ip": {
"type": "ip"
},
"latitude": {
"type": "half_float"
},
"location": {
"type": "geo_point"
},
"longitude": {
"type": "half_float"
}
}
},
"host": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"icmp_code": {
"type": "long"
},
"icmp_type": {
"type": "long"
},
"in_iface": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"metadata": {
"properties": {
"flowbits": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"flowints": {
"properties": {
"applayer": {
"properties": {
"anomaly": {
"properties": {
"count": {
"type": "long"
}
}
}
}
},
"http": {
"properties": {
"anomaly": {
"properties": {
"count": {
"type": "long"
}
}
}
}
},
"tcp": {
"properties": {
"retransmission": {
"properties": {
"count": {
"type": "long"
}
}
}
}
}
}
}
}
},
"payload_printable": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"proto": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"src_geoip": {
"properties": {
"city_name": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"continent_code": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"country_code2": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"country_code3": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"country_name": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"dma_code": {
"type": "long"
},
"ip": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"latitude": {
"type": "float"
},
"location": {
"properties": {
"lat": {
"type": "float"
},
"lon": {
"type": "float"
}
}
},
"longitude": {
"type": "float"
},
"postal_code": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"region_code": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"region_name": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"timezone": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
HAD TO DELETE TO FIT POST
What would i need to do next please?
Hi @bn1980
Sorry that you are struggling.
My first and best suggestion is from my previous post.
Open a new post with a subject containing Suricata / ECS / Mapping etc and tag it with Security. No one is looking at this thread (especially the security folks which I am not ) and as a volunteer I can't really write you a how to, this is a forum not commercial support I provided as much as I can at this point.
In addition there is training materials, free and subscription etc and of course tons of articles and documentation.
This is the world of Open Source / Free Software ... You can of course engage with with professional services / commercially from us or someone else. If you are interested in that you can direct message me.
All that said I would a submit a new post with the right subject line and a well-defined question and tag it with security and you'll probably get some help.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.