Define custom field on launch

Elastic Stack Beats
1

Added two metadata fields under 'host' to our index template and preloaded this into our elastic cluster like this:

    "metadata": {
      "properties": {
        "id1": {
          "type": "long"
        },
        "id2": {
          "type": "long"
        }
      }
    },

Hoped we could define these values when launching the winlogbeat service by adding -E metadata.idX=valX arguments (X=1|2) and then in the YML file do this:

processors:
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_host_metadata:
      netinfo.enabled: true
  - add_fields:
      target: host.metadata
      fields:
        id1: ${metadata.id1?You need to set the metadata.id1 environment variable}
        id2: ${metadata.id2?You need to set the metadata.id2 environment variable}

But we get this error when attempting to launch service:

2019-12-09T10:50:11.266+0100	ERROR	instance/beat.go:916	Exiting: error initializing processors: fail to unpack the add_fields configuration: missing field accessing 'processors.3.add_fields.fields.assetid' (source:'C:\Program Files\WinlogBeat\winlogbeat.yml')

Why?

2

Wonder how define a dictionary for the add_fields processor, doc says:

"The add_fields processor adds additional fields to the event. Fields can be scalar values, arrays, dictionaries, or any nested combination of these."

  - add_fields:
      target: host
      fields:
        metadata: {
          id1: ${metadata.id1?You need to set the metadata.id1 environment variable},
          id2: ${metadata.id2?You need to set the metadata.id2 environment variable}
       }

Just gives me:

Exiting: error loading config file: yaml: line 131: did not find expected ',' or '}'

Need I define environment variables or can I parse as -E metadata.idX=valX arguments?

3

:slight_smile: Seems what failed was the env.var expansion:

${metadata.id1?You need to set the metadata.id1 environment variable}

This works as expected:

  - add_fields:
      target: host
      fields:
        metadata.id1: ${metadata.id1:def.value1}
        metadata.id2: ${metadata.id2:def.value2}

giving me:

host.metadata.id1: env.var.value1
host.metadata.id2: env.var.value2