Question about Security Module + ECS + Sparcity Question

Elastic Stack Beats
1

Hi,
I have some questions regarding the security module.
Using the processor.Convert() the mapping between windows event data into ECS is done by renaming fields.
I have some doubts regarding to the behavior

  • When the original field is exists, but it is null, the destination ECS field is not is not created
    As an example, some instances of event 4625
    image
    image.png1237×230 18.1 KB

winlog.event_data.IpAddress exists for all document but source.ip is only populated when winlog.event_data.IpAddress is not null.

Is this the expected behavior? If yes:

  • Does exists a way to, event when winlog.event_data.IpAddress assing a default or dummy value to the populated field in ECS?
    In this way I can perform allways agregations using the field source.ip; if do not exist I can use for some events 4625 but not for all.

  • Does it better to have "dense" documents? I mean, similar documents ( documents corresponding to event.code 4625) to have all the same fields with data?

Thank you very much
Regards
Ana

2

I was debating whether it should copy or rename from the start. Copying would probably be better in the short term while there are only a few events handled by the module. By leaving the original fields in tact and populating ECS fields you can correlate with other ECS normalized events and also other Windows events that happen to use the same field names.

I propose to add a configuration to allow the behavior to be selected (with a default as copy). The script processor supports params and we could allow users to select rename if they want to make that trade-off. Like

  - name: Security
    processors:
      - script:
          lang: javascript
          id: security
          file: ${path.home}/module/security/config/winlogbeat-security.js
          params:
            mode: rename

How does that sound?

3

Hi Andrew,
It sounds good to rename in order to make the trade off.

The other question was (probably I wasn't able to explain myself properly):
In some cases you have events of the same type under specific conditions or because of the version of the windows can have a value or not
For example the 4625, when connecting via RDP

  • If the target system is windows server 2012, depending of how the authentication is configured and the configuration of the RDP client IP address for the client won't be logged, so the winlog.event_data.IpAddress is null
  • If the target system is windows server 2016 winlog.event_data.IpAddress has the IP address of the client initiating the conection

image
image.png1485×510 38.3 KB

The problem appears,for example, when I build an aggregation. If I use the source_ip in a Term aggregation, events from windows server 2012 won't appear because source.ip field do not exists.
For example

image
image.png1526×214 19 KB

So my original question was if when the field winlog.event_data.IpAddress is null is it possible either map the source.ip and have a null value or to asign a default value (like N/A?) and as a consecuence of this a reflexion: Is it better to have homogeneous 4625 documents from the point of viewof the sparcity? (i.e. that all events 4625 has the field ip.source field)

Thank you
Regards
Ana

4

In a lot of cases this is unnecessary because aggregations can be configured w.r.t. how missing values are handled. If you wanted documents without a particular field to be treated as if they contained "N/A" then you could tell a terms aggregation, for example, to use "N/A" as the missing value (doc ref: missing values). The Kibana terms agg exposes this in the UI.

26%20AM

Sparcity isn't much of a problem since improvements were made in ES 6.x. Space Saving Improvements in Elasticsearch 6.0 | Elastic Blog

5

Thank you Andrew
Regards
Ana

6

Hi Andrew,
This option appears disabled for the source.ip
image
Thank you

7

Solved using {"missing": "1.1.1.1"} in the JSON INPUT

8

Nice! Thanks for updating with that solution.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.