Host.os is missing in generated template when host is expected in a custom schema using ECS Tooling 1.6 and 1.7

Elastic Stack Elasticsearch
#1

Hi, I am using ECS Tooling (1.6 and 1.7) to generate template, adding "host" to a custom schema "Link". Added host.yml as following. In the resulted template.json, while host.geo exists, host.os is missing.

After I changed os.yml file (in buildin "schemas" dir) name to be o.yml, or bo.yml, host.os presents in template.

This happens for both 1.6 and 1.7.

host.yml (in custom-schema dir)

  • name: host
    title: Host
    .....
    type: group
    reusable:
    top_level: true
    expected:
    - Link

=========
Link.yml (in custom-schema dir)

  • name: Link
    title: Link
    group: 2
    description: >
    Link....
    type: group

    fields:

    • name: Name
      level: custom
      type: keyword
      required: true
      description: >
      A distinguishable name for the link.
      example: Frame Relay Cloud_1141289207

    • name: Protocol
      level: custom
      type: keyword
      description: >
      The type of link represented.
      example: ethernet

I also tested changing geo.yml to o.yml, geo will be missing.

Is this config error in my part of something in the Tool?

Any help is very much appreciated.

#2

Welcome to the community @mxu!

I was able to replicate your issue. This issue is happening because Link.* is reusing host.* at the same phase in which host.* is populating its reuses. When Link nests hosts, hosts hasn't had all its reuses added yet. I'm continuing to dig into this a bit further.

As a workaround, you can add the reusable.order attribute to your custom host.yml definitions and set it to 3:

---
- name: host
  title: Host
  type: group
  reusable:
    order: 3
    top_level: true
    expected:
    - Link

The default value is 2, and using 3 will ensure that nestings expected Link will come after the host.* nestings.

#3

Thank you, Eric. I will try reusable: order: 3.
It seems related to the alphabetical order of schema file names . geo.yml is before host.yml, it does not have this problem. After changing os.yml to _os.yml or bs.yml (whatever before host), it would work fine.
Thanks again!

#4

Eric, After adding order: 3, the template was generated correctly. Thanks for your help.
It may be worthy to add that to README?
And please note that "geo" was correctly added to Link....host without "order: 3"

#5

Thanks, @mxu!

You're correct about the alphabetical ordering; each group is sorted alphabetically while the reuses are populated.

I've opened an issue to document reusable.order and note its usage in this type of situation, and I'll open a PR to address.

#6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.