Output to multiple indexes Filebeat 6.5

Elastic Stack Beats
1

Hi,

i am trying to configure filebeat to put data into multiple indexes. That is why I define for every prospector a field "category" (fields.category: myApp) and use this field in the parameters "setup.template.name=filebeat-%{[fields.category]}", "setup.template.pattern=filebeat-%{[fields.category]}-*" and "output.logstash.index=filebeat-%{[fields.category]}". Unfortunately only one index is being created with the name "filebeat-%{[fields.category]}-2019.02.06". Could you please tell me what I am doing wrong?

I am deploying filebeat with kubernetes helm and my filebeat.yml looks like this.
Thank you!

filebeat.config:
  modules:
    path: ${path.config}/modules.d/*.yml
    reload.enabled: true
  prospectors:
    path: ${path.config}/prospectors.d/*.yml
    reload.enabled: true
filebeat.prospectors:
- combine_partial: true
  containers:
    ids:
    - '*'
    path: /var/lib/docker/containers/
    stream: all
  enabled: true
  fields:
    category: myApp
  include_lines:
  - ^[0-9]{4}-[0-9]{2}-[0-9]{2}.*\[custom-regex.*
  multiline:
    match: after
    negate: true
    pattern: ^[0-9]{4}-[0-9]{2}-[0-9]{2}
  processors:
  - add_docker_metadata: null
  - add_kubernetes_metadata:
      in_cluster: true
  type: docker
http.enabled: true
http.port: 5066
logging.level: debug
output:
  file:
    enabled: false
  logstash:
    hosts:
    - logstash:5044
    index: filebeat-%{[fields.category]}
output.file:
  filename: filebeat
  number_of_files: 5
  path: /usr/share/filebeat/data
  rotate_every_kb: 10000
processors:
- add_cloud_metadata: null
setup:
  template:
    name: filebeat-%{[fields.category]}
    pattern: filebeat-%{[fields.category]}-*
2

I am pretty sure you need to use Logstash to do this, beats will only ever send to one index.

3

Hi @warkolm , thank you very much!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.