ECS in avro format

Elastic Observability Logs
1

I'm looking for avro schema representation of ecs. It is necessary in Apache NiFi, when you're trying to transform data with record-oriented processors. Workaround is to use InferAvroSchema processor, but there is necessary some json file with all ecs fields. ECS tooling doesn't provide this

So, is it available somewhere one of these two options, please?

  1. avro schema representation of ecs
  2. example json file with all ecs fields according to ecs version

Thanks a lot

2

Hi, @birko!

There are two YAML files that are a full representation of ECS maintained in the ECS repo. You can view whatever version(s) you need by switching the branch/tag.

These files can be converted to JSON and might be a good starting place for what you need here.

3

Hi @ebeahan,
I did this, maybe it will be helpful for someone:

  1. csv with columns Field,Type,Example from fields.csv (generated by ECS tooling)
  2. some corrections of this csv (many examples were missing)
  3. csv->json conversion via CSV To JSON Converter
  4. InferAvroSchema created avro schema according to json
1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.