Problem connecting filebeat (v. 6.6.0) with ingress pipeline


(Agnes) #1

Problem to connect filebeat version 6.6.0 with ingress pipeline

Done:

  1. created a (custom) ingress pipeline "my_pipeline"
  2. enrolled a new Beat of typ filebeat in may ECE-Deplyment
  3. enrolled filebeat on my server, where the log files are stored
  4. created a tag for the beat with one configuration block of typ "Filebeat Input",
    which contains the location of log files on my server in section "Path".
    I've checked /var/lib/filebeat/registry on the server.
    It show the correct source = defined Paths in the tag.

Where can I configure, tha output.elasticsearch.pipeline: my_pipeline ?
I've tried to enter this in the same configuration block, where the Paths are given in
section "Other config".
After saving the configuration block all seems OK, the String "output.elasticsearch.pipeline: my_pipeline" is saved under
"other config"
But, when I save the whole tag, then the entry under "Other config" ist deleted.

Is this the right place to give the output to pipeline information?


(Agnes) #2

SOLVED:

changes in filebeat.yml:

  • enabled: true

addings to filebeat.yml

paths:
- /tmp/abc.log
fields:
type: applog
project: pro
app: my
fields_under_root: true

paths:
- /tmp/xyz.log
fields:
type: applog
project: pro
app: yours
fields_under_root: true

output.elasticsearch:
hosts:
- 'https://bbb7329222...b900a.our-cluster.our.domain:9243'
protocol: https
indices:
- index: "abc-%{+yyyy.MM}"
when.contains:
app: "my"
- index: "xyz-%{+yyyy.MM}"
when.contains:
app: "yours"
username: elastic
password: ourpassword
pipelines:
- pipeline: "my_log"
when.contains:
app: "my"
- pipeline: "yours_log"
when.contains:
app: "yours"

ECE

removed Beat and tag, couldn't get it to work.