You can wrap a drop filter in a conditional that checks the content of a field. Some examples:
At the moment i have this filter working to drop some lines
## var_log_messages filter
filter {
if [type] == "var_log_messages" {
grok {
patterns_dir => ["/etc/logstash/patterns/"]
match => { "message" => "%{SYSLOGTIMESTAMP:var_log_messages_timestamp} %{SYSLOGHOST:var_log_messages_hostname} %{DATA:syslog_program}(?:\[%{POSINT:var_log_secure_pid}\])?: %{GREEDYDATA:message}" }
overwrite…
logstash
I have a log file and I am parsing it through Logstash and storing it in some place. But the the problem is that some lines in the log file do not always match my grok pattern and are therefore tagged as 'grokparsefailure' etc automatically. I do not tag any line explicitly, so the lines which are automatically tagged by logstash are the wrongly structured lines and I want these lines to be skipped i.e. I don't want these output lines to appear in the output at all (lines which have tags).
Can …