While using kafka input, I want to output only when json data contains a specific string.
I tried setting "include_lines" in filebeat.yml, but it was not filtered properly.
When the filebit.yml setting is as follows and data-set1 and 2 are input, not only data-set1 but also data-set2 are output.
I expected only data-set 1 to be output, but it wasn't.
What did I make a mistake?
- part of the filebeat.yml
filebeat.inputs:
- type: kafka
hosts:
- qa-parkbae-01.hanpda.com:9092,
- qa-parkbae-02.hanpda.com:9092,
- qa-parkbae-03.hanpda.com:9092
topics: ["parkbae-test-topic1"]
group_id: "test123"
ssl.enabled: false
include_lines: ['\"event\":\"basket\"']
-
data-set1
{"id":"parkbae","event":"basket","data":"test1"} -
data-set2
{"id":"parkbae","event":"ball","data":"test2"}