I'm currently working on implementing ECS but i'm stuck with IPS Logs.
As many Antivirus, IPS, HIPS and other detection softwares, my IPS generate logs containing "Application" information and behavior thought network monitoring. Logs about "Application" can contain many usefull information like :
Application Name
Application Category
Application ID
Application Risk
Application Path
...
I can't really find a Field Set where I can put this kind of information. The closer one should be "Service" Field set but for me this information are not really related to a service. I can also put "Application Name" in "network.application" field, but it's not really accurate.
Does someone has ever face this issue ? Could someone give me an advise on that ?
Another similar point is about AV/HIPS/IPS/... Signature, I have information about :
Signature Name
Signature ID
Signature Reference
Signature Payload
...
I plan to use the "Obsever" Field Set for this kind of information and create fields like "observer.signature.name", "observer.signature.id", etc...
Is it really relevant to use this Field Set in for this case ?
Observer Field Set seems to be usefull for my need, who are mainly SIEM oriented. In fact, I get many logs from Proxy, WAF, IDS, IPS, Firewall, HIPS, and so on. I can use Observer Field Set for information concerning those devices themselves, but now what is the best solution for information they provide ?
I thought of two solutions :
Continue to use Observer Fieldset with fields like :
observer.proxy.category.id
observer.proxy.category.name
etc...
Create a Field Set for each Observer Type :
ips.signature.name
proxy.category.id
antivirus.virus.category
etc...
Knowing that I use "file.hash.*", "event.action","event.outcome", etc... regardless of the observer type.
The Observer fields are meant to carry metadata about the device or system doing the monitoring. It's not meant to contain the events generated by these systems, however.
For the events they generate, you should try to use other existing ECS field sets such as event.*, the upcoming threat.* fields (PR in progress) and so on. Note that it's common for an event to be broken up into various fields in different field sets. For example, an event's duration should always be captured in event.duration, even if it's the duration of an HTTP request.
For any event source that doesn't have a clear mapping in ECS (antivirus, proxy, alerts), you should capture them in custom fields*.
When you've worked out a custom schema that works well for your use case, you're more than welcome to submit an issue or pull request to ECS at https://github.com/elastic/ecs, to get them added to ECS. We want to add more fields to ECS, and any community input we get will be invaluable to help shape the future of the schema in a way that makes sense to you
* Note that adding custom fields carries a risk of conflict with future versions of ECS. In order to make this smoother, we generally recommend nesting your fields under a prefix that's unlikely to be used by ECS. For example ECS tries to avoid brand names, and use concept names instead. So you could nest your custom fields under your company name, project name or a third party tool's name, depending on what makes most sense for your situation.
It's good to know that a threat.* field set is upcoming, I will adapt my schema following that and re-adapt when the field set will be released.
Sure, my events fields are distributed on multiple field set in general, I'm more concerned by accuracy in long term and to choose fields that are relevant for analysts, use cases, etc...
When I will be satisfied with my schema, I will take a look at how to submit a pull request to ECS Git.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
