ECS recommended data types

[jancodenew (jan)]

Hi,

What will be the impact if i change the ECS recommended data type for ECS fields. For example what if i map the event.action field as 'text' datatype instead of keyword.

I am receiving the following values in event action.

values: PRofile created, profile CREATED, Profile CreateD etc. I want to query/search this event.action field as case insensitive.

so i want to change the data type from ecs recommended keyword to text type to do analyzed search.

[spinscale (Alexander Reelsen)]

In this concrete example you will need more space and you will not be able to run aggregations when using text as a field type.

As a general rule of thumb, I would try to leave the fields as is (always keep in mind the person coming after ), but you are free to add more fields and just copy the value from one field to another. Two possible solutions here.

First use copy_to in the mapping, see copy_to | Elasticsearch Guide [7.13] | Elastic

Second, use a processor within your ingest pipeline, see Set processor | Elasticsearch Guide [7.13] | Elastic

[jancodenew (jan)]

Thank you #spinscale for your reply.

But my doubt here is, ECS is not recommending any text data type. ECS recommends to maintain all fields in keyword. So when i copy_to a new field with text data type, will this impact the SIEM Security detection or any other areas?

I understand the two issues mentioned above ie. more space consuming and aggregation will not work with text data type fields.

[Christian_Dahlqvist]

You could perhaps copy_to a new keyword field with a lowercase normalizer if you do not want to alter your data through a pipeline. This will index the keyword in lowercased form and provide case insenitivity.

[jancodenew (jan)]

Thank you #Christian_Dahlqvist for your reply.

For a conclusion, Is this ECS not allowing to maintain a single field with data type other than keyword ?

Here I am trying to understand the best practice of using ECS.