Suggestion to improve ECS documentation

[SH_HSOC]

Hi Elastic folks,

I have a proposal to improve your documentation for the Elastic Common Schema. I would suggest to reference the filebeat modules for common log sources in the ECS documentation. E.g. under Additional Information.

I just noticed those after I put some effort in doing my own ECS normalization of PANOS logs via logstash. Would have helped me a great time to borrow from the module documentation and the ingest pipeline to not fully come up with my own mapping.

Anyhow could be a great pointer in the ECS documentation. Kind of like "And here is how we at Elastic would normalize these common log sources". And to really make our lifes a lot easier you could also provide a logstash config doing the same adaption like the corresponding input.yml and pipeline.yml. After all some of us use logstash for the whole log parsing instead of ingest pipelines.

Best regards,
Sebastian

[webmat (Mathieu Martin)]

Thanks for the suggestion. It's a good idea, yes.

While we can't cover in details all data sources that map to ECS, simple pointers like you suggest would indeed be helpful.

Also, I hear you about using Logstash vs Ingest pipelines. Here's a tip (unrelated to your suggestion). When you peer into the Beats modules, some of them have two pipelines: a set of Beats processors, and an Elasticsearch pipeline. If you use Beats to send events to Logstash, you may be able to use the Beats processors as is, and save yourself some work

Here's the Beats processors for the panw module, for example. And here's thoughts I've captured to help people navigate and understand Filebeat modules: https://gist.github.com/webmat/be9d145d52749ce61b5c75621a383f52

Good luck!