Extracting data from logs using Filebeat


#1

Hello dear,
My company is using Scalyr for monitoring logs. Now, i am making the POC using Graylog2.
In Scalyr, there is a scalyr agent that manage logs before send to Graylog.
For example,

{
path: “/var/log/nginx/*.access.log”,
attributes: {parser: “accessLog”, serverType: “nginx”},
sampling_rules: [
{ match_expression: " HTTP/\d+?.\d+?" (?:402|405|410|429|(2|3)[0-9]{2}) ", sampling_rate: 0 }
]
}

sampling_rate: 0, it means the log will not be sent to Scalyr.
I think Filebeat is doing the same thing like the Scalyr agent.

#exclude_lines: ['^DBG']
#include_lines: ['^ERR', '^WARN']

Are those 2 lines the right answer to make Filebeat doing the same thing like Scalyr agent ?
Or do you have other opinions ?
Thank you


#2

Bump :slight_smile:


(Steffen Siering) #3

exclude_lines/include_lines use regular expressions. If you always want to drop by the regex, then exclude_lines seems to be your setting.


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.