How to filter in filebeat yml file

While using kafka input, I want to output only when json data contains a specific string.

I tried setting "include_lines" in filebeat.yml, but it was not filtered properly.

When the filebit.yml setting is as follows and data-set1 and 2 are input, not only data-set1 but also data-set2 are output.

I expected only data-set 1 to be output, but it wasn't.

What did I make a mistake?

  • part of the filebeat.yml
filebeat.inputs:
- type: kafka
  hosts:
    - qa-parkbae-01.hanpda.com:9092,
    - qa-parkbae-02.hanpda.com:9092,
    - qa-parkbae-03.hanpda.com:9092
  topics: ["parkbae-test-topic1"]
  group_id: "test123"
  ssl.enabled: false

  include_lines: ['\"event\":\"basket\"']
  • data-set1
    {"id":"parkbae","event":"basket","data":"test1"}

  • data-set2
    {"id":"parkbae","event":"ball","data":"test2"}

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.