Filebeat5 alpha 5: multiline and procesor not work as expectation


(vi) #1

Hi,
I'm using filebeat alpha 5 installed in my Windows server to flushing log to my ELK. In filebeat.yml, I configure multiline and processor as below:
#I only want to get log line containing ER or ERROR
processors:

  • drop_fields:
    fields: ["IN:", "INFO:", "[IN]", "[INFO]"]

#log lines not begin with number will be join to one line
multiline.pattern: ^[0-9]{4}
multiline.negate: true
multiline.match: after
multiline.max_lines: 5
multiline.timeout: 5s

But the log displayed on Kibana as attached image. That are not my expectation


(Steffen Siering) #2

it seems like you want to drop events, not fields. Check out includes/exclude_lines settings.


(vi) #3

It works fine as you guide. Thanks.
Can you instruct me how to configure rotation of elastic log and data indices? For each server, I have about 600 log files to be scan. I intend to use logstash monitoring logs from many servers. This will leads the log and indices keep huge capacity. So I need configure to delete old Elastic log and indices to free up disk.


(ruflin) #4

To cleanup old indices, you are probably looking for curator: https://www.elastic.co/guide/en/elasticsearch/client/curator/4.1/index.html


(system) #5

This topic was automatically closed after 21 days. New replies are no longer allowed.