Filebeat drop events with multiple words

Elastic Stack Beats
1

I'm trying to drop events with multiple words like

Tuple, TUPLE, tuple

Complete config doesn't seem to work with one word filter too

filebeat:
  registry_file: /var/run/filebeat/.gl-filebeat-registry
  spool_size: 2048
  prospectors:
    -
      paths:
        - /opt/cargovan/storm/apache-storm-1.0.2/logs/workers-artifacts/*/*/worker.log
      input_type: log
      ignore_older: 10m
      close_older: 5m
      max_bytes: 2097152
      fields:
        OO_CLOUD: test-cloud
      fields_under_root: false
      tail_files: false
  processors:
  - drop_event:
      when:
        regexp:
           message: 'TUPLE'
output:
  logstash:
    hosts: ['test-url:5044']
    loadbalance: true
    worker: 2
    max_retries: -1
    bulk_max_size: 2048
logging:
  level: warning
  to_files: false
  to_syslog: true

Also tried using with multiple words like below

  processors:
  - drop_event:
      when:
        regexp:
           message: 'TUPLE'
           message: 'Tuple'
2

You can set a single regexp to match multiple words, like this:

- drop_event:
    when:
      regexp:
        message: 'Tuple|TUPLE|tuple'

However, if what you want is to drop regardless of case, then I suggest prefixing the regexp with (?i), as in:

message: '(?i)tuple'

Have a look at:

https://www.elastic.co/guide/en/beats/filebeat/5.5/regexp-support.html

3

Thank you @adrisr I'm using Filebeat version 5.0 and above doesn't seem to filter any messages with tuple String
Sample which is still flowing

message

2018-01-24 19:42:01.994 o.a.s.d.executor [INFO] Processing received message FOR -2 TUPLE: source: __system:-1, stream: __metrics_tick, id: {}, [60]

4

Interestingly the below solution worked

  prospectors: 
    - 
      close_older: 5m
      exclude_lines: 
        - Tuple
        - TUPLE
        - tuple
5

Glad you found a solution!

6

Thank you for your help!

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.