Need to drop the commented lines and some lines matching the specified string

Elastic Stack Beats
1

I have installed ELK in one server and filebeat in other server where logs resides. My logs are moved and able to view in Kibana. But I dont need the commented lines and lines with certains text to be displayed in kibana. Hence I used drop_event and exclude_lines in Filebeat and I even used drop filter in logstash but I dont see them refelecting in Kibana dashboard. Can anyone help on this

logstash:
filter {
if "#" in [message] {
drop { }
}
}

filebeat:

  • drop_event:
    when:
    contains:
    message: "#"

I tried even exclude lines but nothing is working. Can anyone help on this. Thanks in advance

2

Hi, could you please format your post using code tags? There might be bad indentation somewhere.

3

Hi!! Thanks for your reply. I found the solution by using multiline pattern in Filebeat.yml. It seems multiline will work for input log but not for filestream. The issue which I am facing now is , In Kibana index is not getting created if my change my input as log instead of filestream.
Do we need to give any permissions for logfile? Can you give your input on this.

multiline.pattern: '^['
multiline.negate: true
multiline.match: after

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.