Another Feature Request for SIEM

[lusynda]

Hi,
I was thinking that our SIEM really need a feature to query from a file, like in my use case, i have a list of infected domain that i need to query every 15m, the problems is that typing them by hand into the query box take sometime and frankly it is tedious it would improve my time if we can query a field against a file. That would speed up thing and save me a lot of time to do other thing.

Thanks

[madduck]

Seconding this request.
If I want to check for communication to a set of hosts in multiple detections I have to copy + paste a huge array of IPs and whenever I want to add/remove one I have to open up every single rule and hope I didnt overlook one.
A list would give us a central place to make those sorts of changes.

If a detection could somehow check fields in an index I think this could be realised without adding file lookup support.

[Mike_Paquette]

@lusynda and @madduck Thank you very much for the suggestions. We agree that using lists as part of detection rules would be a very useful feature. This is currently under consideration for a future release, but I can't provide any specifics regarding if/when it might become available.

We see lists being useful in both the detection logic of a rule, and in the exception logic. Exception logic would specify a set of conditions that, if met, would prevent the creation of a detection signal (alert) that otherwise would occur.

For example let's say you wanted to detect users visiting sites where they could download penetration testing tools, your detection rule logic might include a KQL term such as destination.domain:metasploit.com

But you don't want it to generate a detection signal (alert) if a member of your InfoSec staff happens to visit such a site. You'd like to create an exception that logically does user.name: "is in list":InfoSec_team. ( <--This is not actual KQL syntax)

Thus what you'd get is the logical equivalent of "Generate a signal (alert) each time someone visits a pen testing site UNLESS their user name is in my InfoSec list." As you can imagine, being able to keep one copy of the "InfoSec" list, and then use it in an exception for multiple rules would be very handy.

Work on exceptions is described in this GitHub issue: https://github.com/elastic/kibana/issues/65937

@lusynda Can you please tell me a bit more about your list of infected domains?

• Are you looking to create a signal (alert) each time the domain in a received event contains a value that is included in your list?
• Is the list simply a set of values (i.e. a one-column list) or is it a table of multiple columns?
• Approximately how many entries are typically found in the list?
• What format is this list provided in?
• Are you ingesting /can you ingest this list into an Elasticsearch index?
• If you had the ability to add "exceptions" as I've described above, would you expect to add any?

@madduck we like your suggestion of the detection rule accessing an index for this "list" purpose. Your use case seems to be using lists of IP addresses. Would you mind sharing a few more details about this list?

• Are you looking to create a signal (alert) each time an IP Address in a received event contains a value that is included in your list?
• Approximately how many entries are typically found in a list?
• Is the list simply a set of values (i.e. a one-column list) or is it a table of multiple columns?
• Is this list specified as individual IP addresses? ranges? CIDR blocks?
• Does the list contain IPv4, IPv6, or both?
• What format is this list provided in?
• Are you ingesting /can you ingest this list into an Elasticsearch index (per your suggestion)?
• if you had the ability to add "exceptions" as I've describe above, would you expect to add any?

Thanks again for the suggestions, and please keep 'em coming

[lusynda]

Yes to answer your question

• Yes i want to create an alert everytime the domain in my list got query.
• This list that i have is an one columns, each line contains one domain.
• There are a lot, like a lot of line in my file since there are thousand of infected domain. But i know there can be some problems in query a too large data so if it ls ok i want to be able to be at least query up to 1000 line.
• The format is very simple (the [.] are to that they wont be hyperlink to anything the actual format has no [ ].

facebook[.]com
google[.]com

• I dont thinks that i suggested to index the list. I just simply want to query an index against the file.
• Well the exception is something i hasnt really though of, but that is great, i would love to have that in my kibana.

Thanks you.

[Mike_Paquette]

Thank you for sharing your details @lusynda. I will share this feedback with our development team. Thanks again!

[madduck]

@Mike_Paquette

• Are you looking to create a signal (alert) each time an IP Address in a received event contains a value that is included in your list?
• Approximately how many entries are typically found in a list?
• Is the list simply a set of values (i.e. a one-column list) or is it a table of multiple columns?
• Is this list specified as individual IP addresses? ranges? CIDR blocks?
• Does the list contain IPv4, IPv6, or both?
• What format is this list provided in?
• Are you ingesting /can you ingest this list into an Elasticsearch index (per your suggestion)?
• if you had the ability to add "exceptions" as I've describe above, would you expect to add any?

Hey Mike,

thanks for reaching out.

I dont have a single use case where this issue came up but a handful.
I'll try to answer your questions as best as I can.

• Are you looking to create a signal (alert) each time an IP Address in a received event contains a value that is included in your list?

Yes - one of the use cases is to check for connection to known C2 (Command and Control) servers - so any kind of communication is relevant

• Approximately how many entries are typically found in a list?

Highly dependant on the use case. Some entries might have well over a thousand IP Addresses, some lists (privileged users f.ex.) might only have like 20-30

• Is the list simply a set of values (i.e. a one-column list) or is it a table of multiple columns?
• Is this list specified as individual IP addresses? ranges? CIDR blocks?

Again, this depends on the use case. Privileged Accounts might only have two entries: Account Name + Description whereas C2 Lists can range from a single IP Address to IP Addresses with CIDR Blocks. Mainly we are dealing with single IP Addresses here though.
Theres also some lists online (Feodotracker ipblocklist.csv) that contain some extra information (mainly when the IP was added etc.)

• What format is this list provided in?
• Are you ingesting /can you ingest this list into an Elasticsearch index (per your suggestion)?

The example I linked is a .csv, most of these lists can be parsed as a .csv and yes I could potentially upload this information to a Elasticsearch index. As a matter of fact I have created a index in the past containing privileged account names because I wanted to try if that would work.

• if you had the ability to add "exceptions" as I've describe above, would you expect to add any?

For my specific use cases there are rarely "exceptions" because most lists are very concrete and when dealing with singular IP addresses/accounts I would simply remove them from the list/index. However I think having the option to have exceptions wouldn't hurt.