How can i filter errors in filebeat

[Martin_perez (martinelastic)]

Hi, I have an elk stack mounted, which receives data from two filebeats.
I want to configure my filebeat configuration files so that it only collects files whose error is anything other than 500, and I have tried to do a grep on the "path" line, to try to touch the include or exclude lines ... but I keep getting the same logs, it does not exclude or include anything. What can be?

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /server/www/html/*/logs/error.log | grep -v "200" | grep "2021"
    #- c:\programdata\elasticsearch\logs\*

  # Exclude lines. A list of regular expressions to match. It drops the lines that are
  # matching any regular expression from the list.
  exclude_lines: ['200']

  # Include lines. A list of regular expressions to match. It exports the lines that are
  # matching any regular expression from the list.
  include_lines: ['400', '500']

Many thanks in advance, greetings

[mtojek (Marcin Tojek)]

Did you at this thread before: Filebeat exclude line not working :( - #2 by robscott27 ? I think it's similar.

[Martin_perez (martinelastic)]

Yes, also i tried like that post and other many ways
exclude_files: ['*.200.* ']
exclude_files: [' 200 ']

but, neither of them works, filebeat is working but i don´t get logs

[mtojek (Marcin Tojek)]

Hm.. did you try the directive exclude_files or exclude_lines?

[Martin_perez (martinelastic)]

this is my config file, and I receive all kinds of logs, that is, the exclude_lines or include_lines lines are not affecting me

  paths:
    - /server/www/html/*/logs/*.log | grep "27/May"
  exclude_lines: ['^200']
  exclude_files: ['^200']

the type of logs on kibana that I wish they did not appear are like this

192.168.x.x - - [27/May/2021:10:05:45 +0200] "GET /sistemas/noticias/x HTTP/1.1" 200 879 "http://x/sistemas/noticias/x" "Mozilla/5.0 (X11; Linux armv7l)x/537.36 (KHTML, like Gecko) Raspbian x/65.0.3325.181 x/65.0.3325.181 x/537.36"