# Openstack log filter **URL:** \ **Category:** Logstash\ **Created:** [May 23, 2019, 4:06pm UTC](https://discuss.elastic.co/t/openstack-log-filter/182487 "2019-05-23T16:06:00Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![DavideG](https://avatars.discourse-cdn.com/v4/letter/d/49beb7/32.png) [@DavideG](https://discuss.elastic.co/u/DavideG)\ **Post date:** [May 23, 2019, 4:06pm UTC](https://discuss.elastic.co/t/openstack-log-filter/182487/1 "2019-05-23T16:06:00Z")
Hello guys, I'm having a problem where I have been stuck for the past 3 days if you would be able to help me I would appreciate very much 😃 I'm trying to filter logs coming from openstack more precisely from the nova service, but I want to ignore logs that came with the word DEBUG or INFO, so, up untill now this is my configuration: /etc/logstash/conf.d/01-logstash-beats-input.conf: input { beats { port =\> 5044 ssl =\> true ssl\_certificate =\> "/etc/pki/tls/certs/logstash.crt" ssl\_key =\> "/etc/pki/tls/private/logstash.key" } } /etc/logstash/conf.d/15-openstack-filter.conf filter { grok { match =\> ["message", "%{DATE:date} %{TIME:time} %{POSINT:openstack\_pid} %{LOGLEVEL:loglevel}"] patterns\_dir =\> ["/etc/logstash/conf.d/patterns"] } } /etc/logstash/conf.d/30-elasticsearch-output.conf output { elasticsearch { hosts =\> "localhost:9200" index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}" } } The grok file: /etc/logstash/conf.d/patterns/grok OPENSTACK\_PID ( %{POSINT:pid:int})? LOGLEVEL ([A-a]lert|ALERT|[A-a]udit|AUDIT|[T|t]race|TRACE|[D|d]ebug|DEBUG|[N|n]otice|NOTICE|[W|w]arn?(?:ing)?|WARN?(?:ING)?|[E|e]rr?(?:or)?|ERR?(?:OR)?|[C|c]rit?(?:ical)?|CRIT?(?:ICAL)?|[F|f]atal|FATAL|[S|s]evere|SEVERE|EMERG(?:ENCY)?|[Ee]merg(?:ency)?) Example of logs: 2019-05-23 11:23:08.623 19643 INFO nova.api.openstack.placement.requestlog [req-70f52a92-06a1-44b5-8a3a-268d0d8fac3b 8f5dcc068c684184b31f45ea62e09d74 cd495fc09f024f38aba7565fe1041b50 - default default] 192.0.2.1 "GET /placement/resource\_providers/a9714c16-c34b-46a8-95dd-8dd28ec1ba33/inventories" status: 200 len: 514 microversion: 1.0 2019-05-23 11:23:35.008 5478 ERROR oslo\_db.sqlalchemy.engines File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 856, in query 2019-05-23 11:23:35.008 5478 ERROR oslo\_db.sqlalchemy.engines self.\_affected\_rows = self.\_read\_query\_result(unbuffered=unbuffered) Up untill now Kibana is showing me everything, ERROR INFO, everything when I'm trying to ignore at least the logs with INFO LOGLEVEL Can anyone please help me? Thank you Best regards ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [May 23, 2019, 5:22pm UTC](https://discuss.elastic.co/t/openstack-log-filter/182487/2 "2019-05-23T17:22:32Z")
If you want to discard the events with some log levels you could use ``` if [loglevel] in ["INFO", "DEBUG"] { drop {} } ``` ---
**Author:** ![DavideG](https://avatars.discourse-cdn.com/v4/letter/d/49beb7/32.png) [@DavideG](https://discuss.elastic.co/u/DavideG)\ **Post date:** [May 27, 2019, 1:59pm UTC](https://discuss.elastic.co/t/openstack-log-filter/182487/3 "2019-05-27T13:59:39Z")
Hello again, I'm sorry to insist on this and let me just add that it worked. But now I have another problem, I have another machine sending log messages to logstash with INFO that I want to register, so I was thinking if there is a way to filter this messages on the client machine using Filebeat (I think it makes more sense to filter in client and send less "garbage" I tried this on "filebeat.yml": # Exclude lines. A list of regular expressions to match. It drops the lines that are # matching any regular expression from the list. #exclude\_lines: ['^DBG'] processors: - drop\_event: when: regexp: message: "INFO | DEBUG" It doesn't work The idea is to drop INFO or DEBUG messages Thank you again for the help @Badger ---
**Author:** ![DavideG](https://avatars.discourse-cdn.com/v4/letter/d/49beb7/32.png) [@DavideG](https://discuss.elastic.co/u/DavideG)\ **Post date:** [May 28, 2019, 11:02am UTC](https://discuss.elastic.co/t/openstack-log-filter/182487/4 "2019-05-28T11:02:58Z")
I resolved this with # Exclude lines. A list of regular expressions to match. It drops the lines that are # matching any regular expression from the list. #exclude\_lines: ['^DBG'] processors: - drop\_event: when: regexp: message: "INFO|DEBUG" Thank you very much for the help ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 25, 2019, 11:02am UTC](https://discuss.elastic.co/t/openstack-log-filter/182487/5 "2019-06-25T11:02:58Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.