# Question about Security Module + ECS + Sparcity Question

**URL:** <https://discuss.elastic.co/t/question-about-security-module-ecs-sparcity-question/190957>\
**Category:** Beats\
**Tags:** winlogbeat\
**Created:** [July 17, 2019, 10:24am UTC](https://discuss.elastic.co/t/question-about-security-module-ecs-sparcity-question/190957 "2019-07-17T10:24:42Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Anabella\_Cristaldi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anabella_cristaldi/32/23612_2.png) [@Anabella\_Cristaldi](https://discuss.elastic.co/u/Anabella_Cristaldi)\
**Post date:** [July 17, 2019, 10:24am UTC](https://discuss.elastic.co/t/question-about-security-module-ecs-sparcity-question/190957/1 "2019-07-17T10:24:42Z")

</div>

Hi,  
I have some questions regarding the security module.  
Using the processor.Convert() the mapping between windows event data into ECS is done by renaming fields.  
I have some doubts regarding to the behavior

- When the original field is exists, but it is null, the destination ECS field is not is not created  
As an example, some instances of event 4625  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/4/54fef797c3aaa88b19d0a69bc77ff46c5b7407c3.png)

winlog.event\_data.IpAddress exists for all document but source.ip is only populated when winlog.event\_data.IpAddress is not null.

Is this the expected behavior? If yes:

- Does exists a way to, event when winlog.event\_data.IpAddress assing a default or dummy value to the populated field in ECS?  
In this way I can perform allways agregations using the field source.ip; if do not exist I can use for some events 4625 but not for all.

- Does it better to have "dense" documents? I mean, similar documents ( documents corresponding to event.code 4625) to have all the same fields with data?

Thank you very much  
Regards  
Ana

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [July 18, 2019, 4:20pm UTC](https://discuss.elastic.co/t/question-about-security-module-ecs-sparcity-question/190957/2 "2019-07-18T16:20:39Z")

</div>

> [@Anabella\_Cristaldi](#):
>
> Using the processor.Convert() the mapping between windows event data into ECS is done by renaming fields.  
> I have some doubts regarding to the behavior

I was debating whether it should copy or rename from the start. Copying would probably be better in the short term while there are only a few events handled by the module. By leaving the original fields in tact and populating ECS fields you can correlate with other ECS normalized events and also other Windows events that happen to use the same field names.

I propose to add a configuration to allow the behavior to be selected (with a default as `copy`). The script processor supports params and we could allow users to select rename if they want to make that trade-off. Like

```auto
  - name: Security
    processors:
      - script:
          lang: javascript
          id: security
          file: ${path.home}/module/security/config/winlogbeat-security.js
          params:
            mode: rename

```

How does that sound?

---

<div class="post-metadata">

**Author:** ![Anabella\_Cristaldi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anabella_cristaldi/32/23612_2.png) [@Anabella\_Cristaldi](https://discuss.elastic.co/u/Anabella_Cristaldi)\
**Post date:** [July 19, 2019, 8:55am UTC](https://discuss.elastic.co/t/question-about-security-module-ecs-sparcity-question/190957/3 "2019-07-19T08:55:12Z")

</div>

Hi Andrew,  
It sounds good to rename in order to make the trade off.

The other question was (probably I wasn't able to explain myself properly):  
In some cases you have events of the same type under specific conditions or because of the version of the windows can have a value or not  
For example the 4625, when connecting via RDP

- If the target system is windows server 2012, depending of how the authentication is configured and the configuration of the RDP client IP address for the client won't be logged, so the winlog.event\_data.IpAddress is null
- If the target system is windows server 2016 winlog.event\_data.IpAddress has the IP address of the client initiating the conection

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/a/d/ad9fcb89abe973907331094a61db009a92242ec7.png)

The problem appears,for example, when I build an aggregation. If I use the source\_ip in a Term aggregation, events from windows server 2012 won't appear because source.ip field do not exists.  
For example

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/2/3/2399b977b435a35268a3674b65a386666561f8de.png)

So my original question was if when the field winlog.event\_data.IpAddress is null is it possible either map the source.ip and have a null value or to asign a default value (like N/A?) and as a consecuence of this a reflexion: Is it better to have homogeneous 4625 documents from the point of viewof the sparcity? (i.e. that all events 4625 has the field ip.source field)

Thank you  
Regards  
Ana

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [July 22, 2019, 1:17pm UTC](https://discuss.elastic.co/t/question-about-security-module-ecs-sparcity-question/190957/4 "2019-07-22T13:17:51Z")

</div>

> [@Anabella\_Cristaldi](#):
>
> So my original question was if when the field winlog.event\_data.IpAddress is null is it possible either map the source.ip and have a null value or to asign a default value (like N/A?)

In a lot of cases this is unnecessary because aggregations can be configured w.r.t. how missing values are handled. If you wanted documents without a particular field to be treated as if they contained "N/A" then you could tell a terms aggregation, for example, to use "N/A" as the missing value (doc ref: [missing values](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html#_missing_value_14)). The Kibana terms agg exposes this in the UI.

![26%20AM](https://us1.discourse-cdn.com/elastic/original/3X/5/2/52f89181b8583457362b9e60264bab36babdf5ad.png)

Sparcity isn't much of a problem since improvements were made in ES 6.x. [Space Saving Improvements in Elasticsearch 6.0 | Elastic Blog](https://www.elastic.co/blog/minimize-index-storage-size-elasticsearch-6-0)

---

<div class="post-metadata">

**Author:** ![Anabella\_Cristaldi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anabella_cristaldi/32/23612_2.png) [@Anabella\_Cristaldi](https://discuss.elastic.co/u/Anabella_Cristaldi)\
**Post date:** [July 22, 2019, 1:43pm UTC](https://discuss.elastic.co/t/question-about-security-module-ecs-sparcity-question/190957/5 "2019-07-22T13:43:38Z")

</div>

Thank you Andrew  
Regards  
Ana

---

<div class="post-metadata">

**Author:** ![Anabella\_Cristaldi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anabella_cristaldi/32/23612_2.png) [@Anabella\_Cristaldi](https://discuss.elastic.co/u/Anabella_Cristaldi)\
**Post date:** [July 22, 2019, 1:50pm UTC](https://discuss.elastic.co/t/question-about-security-module-ecs-sparcity-question/190957/6 "2019-07-22T13:50:27Z")

</div>

Hi Andrew,  
This option appears disabled for the source.ip  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/e/c/ec494b4fb7b12b7010465ae275f606cb8a243dba.png)  
Thank you

---

<div class="post-metadata">

**Author:** ![Anabella\_Cristaldi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anabella_cristaldi/32/23612_2.png) [@Anabella\_Cristaldi](https://discuss.elastic.co/u/Anabella_Cristaldi)\
**Post date:** [July 26, 2019, 11:29am UTC](https://discuss.elastic.co/t/question-about-security-module-ecs-sparcity-question/190957/7 "2019-07-26T11:29:16Z")

</div>

Solved using {"missing": "1.1.1.1"} in the JSON INPUT

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [July 30, 2019, 1:47am UTC](https://discuss.elastic.co/t/question-about-security-module-ecs-sparcity-question/190957/8 "2019-07-30T01:47:16Z")

</div>

Nice! Thanks for updating with that solution.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 27, 2019, 1:47am UTC](https://discuss.elastic.co/t/question-about-security-module-ecs-sparcity-question/190957/9 "2019-08-27T01:47:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.