SOLVED - Index Template for filebeat - fails on fields

[Wayne_Taylor]

Hi,

Recently we switched from beats -> ES Cloud to beats -> logstash -> ES cloud. In our new cluster we found that the index mappings were not done (per previous post).

I am trying to run the commands per documentation Load the Elasticsearch index template | Filebeat Reference [8.11] | Elastic and getting the following error:

Exiting: error unpacking config data: missing field accessing 'fields.version' (source:'/etc/filebeat/filebeat.yml')

Below is the filebeat.yml file.

keystore.path: "${path.config}/filebeat.keystore"

filebeat.config.inputs:
  enabled: true
  path: ${path.config}/configs/*.yml
  reload.enabled: true
  reload.period: 10s
  exclude_files: ['.gz$']

 multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
 multiline.negate: true
 multiline.match: after


#============================= Filebeat modules ===============================

filebeat.config.modules:
  # Glob pattern for configuration loading
  path: ${path.config}/modules.d/*.yml

  # Set to true to enable config reloading
  reload.enabled: true

  # Period on which files under path should be checked for changes
  #reload.period: 10s

#==================== Elasticsearch template setting ==========================

setup.template.settings:
  index.number_of_shards: 1
  index.codec: best_compression
  #_source.enabled: false

#================================ General =====================================

processors:
- add_cloud_metadata: ~
- add_host_metadata:
    netinfo.enabled: true

fields:
  env: "${CLOUD_ENVIRONMENT}"
  version: "${ARTIFACT_VERSION}"
  commit_sha: "${BUILD_TRIGGER_GIT_COMMIT}"
  app_name: "${CLOUD_APP}"
  project_name: "${CLOUD_APP_GROUP}"
  ami_id: "${CLOUD_AMI_ID}"

fields_under_root: true

output.logstash:
  # The Logstash hosts
  hosts: ["myserver:5044"]

Any recommendations?

[jsoriano (Jaime Soriano)]

Hi @Wayne_Taylor,

This error can happen with your configuration if ARTIFACT_VERSION environment variable is not set. Could you check that?

[Wayne_Taylor]

@jsoriano - the variable is available. See attached screenshots to assist.

Wayne

[jsoriano (Jaime Soriano)]

Check if they are also exported.

export | grep BUILD_TRIGGER_GIT_COMMIT

If they are not, export them before running filebeat.

export BUILD_TRIGGER_GIT_COMMIT

[Wayne_Taylor]

@jsoriano - thank you that worked. Did i miss something in the docs for this OR something i can contribute back in the docs?