Hello,
I am trying to test some features of the SIEM integrated in ELK but I am stuck on the rules. In fact I am trying to put a rule where a alert is sent when a specific ip receives packets from another specific ip. I use the Indicator Match rule type and the problem occurs when I try to add the indicator mapping, no matter what I put in the left box it keeps telling me "There aren't any options available" in the right box. See the image.
Could you tell how to resolve this issue please ?
I am a beginner on the ELK stack and I am pretty sure my issue is easy to resolve but I have been working on it and I can't find by myself.
Have a nice day
Check your Indicator Index patterns of logs-ti_* and make sure it exists. Did you want an underscore or hyphen in the name?
Make sure it exists and make sure that it has fields within it that are compatible with destination.ip or source.ip such as another ip field within it.
Hello,
Thanks for your answer.
The logs-ti_* was there by default, I didn't choose the name.
What is the Indicator Index Patterns and logs-ti_* ? Cause I receive suricata logs from another pc and I would like the rule to be applied within those logs, do I have to specifiy it ?
Thank you again for the time you take to answer me.
It chooses a default name but you have to populate different threat intelligence indexes. For example, you can use a filebeat threat intel module like so:
And here are some docs about the threat intel rule:
If you want you can use any index, not just logs-ti_* to do matches but you have to be careful that you're not matching 100k or more threats. A lot of people will make use of the threat query to carefully only query a subset of threat intelligence. If you query a lot then you will end up having a bad time with performance.
Depending on your setup and your company and size of your Elasticsearch though, you can scale it up though, just a tip to not overload your system with a lot of threat intelligence to start out with.
What is the Indicator Index Patterns and logs-ti_* ?
The indicator index pattern is where you match indexes that contain threat intelligence either you have provided custom, through a 3rd party, or through a module from beats. It choose logs-ti_* by default as it assumes you are using filebeat but you can use anything else you want.
Cause I receive suricata logs from another pc and I would like the rule to be applied within those logs, do I have to specifiy it ?
I don't think you want to use suricata logs as a form of threat intelligence but rather just as logs from your network security module (NSM). That doesn't mean you can't grep through Suricata logs to create a threat intelligence list from it but that's not really how it works. Usually you will use a "curated list of intelligence" such as the ones listed in the links above that are free or a special 3rd party one.
If you start to see odd behaviors from Suricata you can either create individual rules to match against those logs to detect things or use our pre-packaged rules in which a lot should be matching against Suricata since most things are ECS compliant:
So most of our rules are ECS compliant meaning that it doesn't matter if they're coming from Suricata logs or if they're coming from audit logs we try to put similar data into similar fields such as source.ip and destination.ip in most cases so you can write detection rules across different log file types. That's how most of the pre-packaged/elastic rules are written as you can see from the repo where they're written here:
If you're new to everything Elasticstack (that's our real name where ELK is kind of a deprecated name since we have much much more than just Elasticsearch, Logstash, Kibana), what I would do is I would go through the pre-packaged elastic rules first and try turning some of those on and seeing if you can write maybe a "fake log" or two that would be detected by them or maybe clone one or two of them and change them slightly and run them first to get a better feel for how they work.
I would use the threat intelligence rule last as you get comfortable sending logs and writing detections and turning on modules.
Good luck though 
and welcome to detection rules by Elasticsearch. I hope you have fun with them.
Hi,
Ok I see.
But I don't understand how can I put the Suricata logs I receive in a curated list of intelligence ?
Because I only receive Suricata logs and nothing else, so I have to manage to create alert rules on those logs.
Do you mean that if I want to use suricata logs directly, I have to create a Indicator Index Pattern based on all the Suricata logs I receive ?
Thanks
Have a nice day
Heya @kesako 
If you'd like to match against IPs in Suricata logs directly, you'd first need to identify where these logs are stored in your cluster. Depending on the way you ingest them, it could be:
filebeat-*indices if you're using good old Filebeat and Suricata modulelogs-*or I guesslogs-suricata.eve-*if you're using Suricata integration- any other index pattern if you customized it
Put the identified pattern in the Indicator index patterns input field.
If your Suricata events are stored in a shared index along with events from other sources (for example, it would be the case for Filebeat), you'd additionally need to specify Indicator index query to filter them.
Please do keep in mind that if you have a lot of events in your Suricata logs, you will most likely experience significant issues with performance when trying to match against them. @Frank_Hassanabad mentioned that:
If you want you can use any index, not just
logs-ti_*to do matches but you have to be careful that you're not matching 100k or more threats. A lot of people will make use of the threat query to carefully only query a subset of threat intelligence. If you query a lot then you will end up having a bad time with performance.
Also, please make sure you do want to use your Suricata logs as a source of threat intelligence data because as far as I understand it wasn't designed for that use case. Again, like @Frank_Hassanabad said,
I don't think you want to use suricata logs as a form of threat intelligence but rather just as logs from your network security module (NSM).
If you still want to do that, one way would be to aggregate your data (IPs, etc) in Suricata logs with ES aggregations (like the terms one) and index it to a separate index like my-suricata-ti. You'd need to make sure to create it with correct mappings, e.g. map your IPs as ip field type. After creating the index and indexing your TI data to it, you could use it in your rule as the indicator index pattern.
Let us know if it helps!
Hey, thanks for your answer.
I did what you specified, and I manage to create the rule but when I do a Ping flood, the rule doesn't trigger and don't give any alert.
Here is a screenshot from suricata events where you can see that in fact the pings are detected.
And the dashboard of the rule.
I don't understand because I specified the index pattern where the rule should work but it still doesn't trigger.
Have a nice day.
I am afraid you are going down kind of a rabbit hole trying to use Indicator match for something it is not intended for while you are still new to detection rules. If you wanted to, you can do something like a ping flood and use a EQL Rule to match against those Suricata external alerts that you found or you could use a simple custom query rule to match against a set of Suricata external alerts.
Suricata does network detections and reports its findings with its own rule set that you tune. If you want to do a "promotion" of the events Suricata found you might be best off for example just using a Custom Query against the logs-suricata-eve-* index with a simple query which matches what you want to find.
For example to promote all signals found from Suricata, it would be this for custom query rule:
suricata.eve.alert.sid: *
Or if you wanted to promote only 1 rule with a sid (signature id) such as 2008124 you would do this query:
suricata.eve.alert.sid: 2008124
I would start with the simpler rule types and become comfortable with creating alerts from the Suricata indexes using them see how many of your use cases that solves for you.
Then later, after you're comfortable with the simpler rules, incorporate the beats module to pull in threat lists as mentioned in the links above and use those threat lists to match those threat lists against indexes to see what bad actors/ip's you want to match against.
Hi, I understand Indicator Match is not the best way for my case.
I tried what you specified but I am still left with no result, in face I made this rule in custom query according to what you said :
Because I had the feeling that maybe I should specify the elastic agent index pattern since it's him who retrieves logs from suricata, but it still gave me nothing.
I did multiple brute force SSH attacks and I made a custom rule in suricata who alerts for this, I can see that it gets the alert in kibana but nothing happens on the rule.
And by the way here is my suricata rule, I'm not sure it helps but here it is:
alert tcp any any -> 172.17.0.3 22 (msg:"Possible SSH brute forcing!"; threshold: type both, track by_src, count 2, seconds 60; sid:10000001; rev: 1;)
Have a good day.
Just take a step back and look at what that custom rule is doing for a second. It's not tricky to hopefully sleuth this out. The detection engine is really a simple design. It basically does queries against index patterns over a time range and if the queries return results then those become alerts/signals.
For Index patterns you're using a lot of patterns but they're overlapping with each other. That doesn't hurt anything but logs-* will end up also matching logs-suricata-eve-* and logs-endpoint.alerts-*, and so on so forth...
Typically people only want to match a few index patterns. For example, if logs-suricata.eve-* is the only index that has the mapping and data of suricata.eve.alert.sid : * then really that should be the only one to specify.
Next, always make sure using the preview button that there are hits and records within the index and the query. An even simpler query to get matches for example would be *:* which means match everything in the index. You can't really go wrong just with *:* and the suricata index if you're trying to match everything. If preview isn't showing anything then I would check your index to make sure it exists and permissions are ok.
Once you start getting matches, then you can tweak it and try more specific things. Going too far when starting out without just trying out simple matching can lead to having multiple issues that prevents any signals from showing up. It's usually best to start out getting a lot of matches in a specific single index first and then expanding your indexes and/or making more concentrated querying.
Good luck
Hey,
I managed to make my rule work, in fact custom query is way more easy and efficient for this particular usage than indicator match.
And in my case suricata.eve.alert.sid didn't match any events but
suricata.eve.alert.signature_id did, I don't know why.
So I can do rather simple rules now but I would like to do a rule for alerting on brute force attack on SSH and it don't seem to have these options available in custom query.
For now I have a custom rule is Suricata that detects it and I just put a rules in ELK to match the sid but I don't think it's the most efficient way.
So yeah I feel like it's a bit limited in terms technical functionality.
And in my case
suricata.eve.alert.siddidn't match any events but
suricata.eve.alert.signature_iddid, I don't know why.
You should explore the index to see what fields are mapped and not mapped to see why you might be seeing this. Typically if it can't match something it's because the mapping is not there in the source index.
So I can do rather simple rules now but I would like to do a rule for alerting on brute force attack on SSH and it don't seem to have these options available in custom query.
You would have to be more explicit about how you would want it to alert on this. There's a lot of flexibility with the rule types to handle a variety of use cases. We would like to know which use cases don't work or give advice on efficiency.
I am looking in the index and I see that suricata.eve.alert.sid is mapped.
But it is marked "ignore_above" in the mapping json.
Does this mean it's ignored in the search ? I would be contradictory with what I saw in the first screen but could explain why I don't have any result is the query.
In this rule I would like to trigger a warning when a SSH brute force attack is detected, so for example when there are more than 5 attempts to connect with a bad password within 10 seconds. I want to do the same thing that my rules in Suricata does (alert tcp any any -> 172.17.0.3 22 (msg:"Possible SSH brute forcing!"; threshold: type both, track by_src, count 2, seconds 60; sid:10000001; rev: 1;)) but in ELK, because I prefer having the rule at one place rather than on all the pc I monitor.
Hi @kesako
I think the Elastic Security "Threshold" Rule type might be a better match for what you are trying to accomplish. If you have network data being ingested into your Elastic SIEM, then you can create a custom "Threshold" rule that is close to what your Suricata rule is doing. Like this:
You could run this rule with an interval of 1 minute to start, then you could tune it by adjusting the threshold value and the interval to achieve the detection you desire.
Additionally, if you have host process data being ingested into your Elastic SIEM (like from auditbeat or the Elastic Endpoint Security Integration for Elastic Agent), then there is already a prebuilt detection rule that detects SSH Brute Force for you.
Please let us know if the "Threshold" rule type works for you.
Hey @Mike_Paquette , thanks for your answer.
I tried your method but I got this error :
Is this normal ? I did a SSH bruteforce attack just before so the preview should give me some elements in the preview.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.