Hello.
I am used to the very basic concepts of Elasticsearch but now facing an issue where I don't know how to proceed. I am indexing data in a fixed format and I am not able to change the format at the source. Now I want to use these data with SIEM. I know about ECS but I am not sure how to convert the data. As far as I understand index template are only usefull to map field types but not to restructure documents. Is that correct? I read about processors and pipelines. Is a pipeline combined with a processor, such as grok, the way to go? How to do that? Would that be applicable to already indexed data or just to new data?
Btw: I already indexed test data with filebeat to see if SIEM works in general. It did. I also know that SIEM doesn't recognize custom indices by default.
Thanks in advance.
Welcome to our community! 
Yes, that's right.
Your best best it to use the Grok Debugger in Kibana to build yourself a pattern, or just play with dissect if it's a simple format. Then expand from there.
Good hints. I will have a look at that. Thank you!
An experimental tool called ecs-mapper might also be useful. ECS-mapper takes a CSV file of your data source's custom fields mapped to ECS fields and helps generate starter pipeline configurations
1 Like
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.