How to customize the exported fields?

Elastic Stack Beats
1

I have a very basic filebeats.yml configuration file:

filebeat:
  inputs:
  - type: filestream
    id: my-log-stream
    paths:
      - /path/to/application/logs/*.log
    json:
      keys_under_root: true
      add_error_key: true
output:
  elasticsearch:
    hosts: [ "https://...." ]
    username: "xxx"
    password: "xxx"
    index: "voucher-app-logs-%{[agent.version]}-%{+yyyy.MM.dd}"
setup:
  template:
    name: "voucher-app-logs"
    pattern: "voucher-app-logs-*"

When filebeat start to forward logs to elasticsearch, it creates a datastream called voucher-app-logs-8.7.0-2023.04.18 and then an index called .ds-voucher-app-logs-8.7.0-2023.04.18-2023.04.18-000001.

If I inspect the index:

curl --location --request GET 'https://my-elasticsearch-domain.com/.ds-voucher-app-logs-8.7.0-2023.04.18-2023.04.18-000001'

I found that it contains many, many, many properties. If I get a document, there are much less fields, because most of the fields are not sent from filebeat to elasticsearch, but they are mapped in the index.

I want just a very little subset of the fields to be mapped in the index.

I saw in the docs that I can do this by creating a custom fields.yml file and point to this file in my configuration, but is this the only way?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.