# Assistance with my Logstash filter **URL:** \ **Category:** Logstash\ **Created:** [October 7, 2020, 9:15pm UTC](https://discuss.elastic.co/t/assistance-with-my-logstash-filter/251342 "2020-10-07T21:15:18Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![HelpComputer](https://avatars.discourse-cdn.com/v4/letter/h/ac8455/32.png) [@HelpComputer](https://discuss.elastic.co/u/HelpComputer)\ **Post date:** [October 7, 2020, 9:15pm UTC](https://discuss.elastic.co/t/assistance-with-my-logstash-filter/251342/1 "2020-10-07T21:15:18Z")
Hello, I am trying to use a variation of the filter provided here - [https://github.com/bromiley/olaf/blob/master/logstash/o365.config](https://github.com/bromiley/olaf/blob/master/logstash/o365.config) but I keep getting the following error, **[FATAL][logstash.runner] The given configuration is invalid. Reason: Expected one of [\t\r\n], "#", "=\>" at line 17, column 12 (byte 616) after filter {** Below is the filter config I am using. Any suggestions on how to correct this? Thanks! ```auto filter { if [log][file][path] == "/var/log/audit.log" { kv { include_keys => ["VlogRecNo", "VigilRecNo", "Pid", "TimeStamp", "Type", "Event", "LinuxPath", "netAddr_IPv4", "Dn", "UserDn", "PATH Type", "Comm"] } if [Comm] == "smdrd" { drop { } } } if ([type] == "o365_csv") { csv { columns => ["PSComputerName", "RunspaceId", "PSShowComputerName", "RecordType", "CreationDate", "UserIds", "Operations", "AuditData", "ResultIndex", "ResultCount", "Identity", "IsValid", "ObjectState"] skip_header => "true" if ([message =~ /^#/) { drop {} } } date { match => ["CreationDate", "ISO8601"] } json { source => "AuditData" } } } ``` ---
**Author:** ![HelpComputer](https://avatars.discourse-cdn.com/v4/letter/h/ac8455/32.png) [@HelpComputer](https://discuss.elastic.co/u/HelpComputer)\ **Post date:** [October 7, 2020, 9:37pm UTC](https://discuss.elastic.co/t/assistance-with-my-logstash-filter/251342/2 "2020-10-07T21:37:46Z")
Realized it was missing a square bracket after message (see below). Added the missing bracket but it's still throwing the same error ```auto if ([message] =~ /^#/) { ``` ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [October 7, 2020, 10:49pm UTC](https://discuss.elastic.co/t/assistance-with-my-logstash-filter/251342/3 "2020-10-07T22:49:22Z")
Could be missing the } to close a previous input, output, or filter section. What do the first 18 lines of the configuration look like? ---
**Author:** ![HelpComputer](https://avatars.discourse-cdn.com/v4/letter/h/ac8455/32.png) [@HelpComputer](https://discuss.elastic.co/u/HelpComputer)\ **Post date:** [October 8, 2020, 1:39pm UTC](https://discuss.elastic.co/t/assistance-with-my-logstash-filter/251342/4 "2020-10-08T13:39:54Z")
Thanks for the reply @Badger! That's the whole filter file posted above. I copied and pasted the contents from above into [http://www.yamllint.com/](http://www.yamllint.com/) and it says it's valid. The error has me baffled. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [October 8, 2020, 2:35pm UTC](https://discuss.elastic.co/t/assistance-with-my-logstash-filter/251342/5 "2020-10-08T14:35:33Z")
> [@HelpComputer](#): > > That's the whole filter file posted above. What other files are part of the configuration? ---
**Author:** ![HelpComputer](https://avatars.discourse-cdn.com/v4/letter/h/ac8455/32.png) [@HelpComputer](https://discuss.elastic.co/u/HelpComputer)\ **Post date:** [October 8, 2020, 4:31pm UTC](https://discuss.elastic.co/t/assistance-with-my-logstash-filter/251342/6 "2020-10-08T16:31:43Z")
The input/output section is another file in that same folder. Password has been removed on post. ```auto input { tcp { port => 5544 } beats { port => 5044 ssl => false } } output { # if [@metadata][beat] == "filebeat" and [input][type] == "netflow" { # elasticsearch { # hosts => ["elastic1", "elastic2", "elastic3"] # index => "netflow-%{[@metadata][version]}" # user => "service-account" # password => "" # } # } if [@metadata][beat] == "filebeat" { elasticsearch { hosts => ["elastic1", "elastic2", "elastic3"] index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}" user => "service-account" password => "" ssl => "true" } } if [@metadata][beat] == "winlogbeat" { elasticsearch { hosts => ["elastic1", "elastic2", "elastic3"] index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}" user => "service-account" password => "" ssl => "true" } } # else { # elasticsearch { # hosts => ["elastic1", "elastic2", "elastic3"] # index => "syslog-%{+YYYY.MM.dd}" # user => "service-account" # password => "" # } # } } ``` ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [October 8, 2020, 4:40pm UTC](https://discuss.elastic.co/t/assistance-with-my-logstash-filter/251342/7 "2020-10-08T16:40:37Z")
Try running with --log.level debug --config.debug and see what it is trying to load. Show us the first 18 lines of the configuration that it loads, you can redact anything you need to, but do not add or delete any lines. ---
**Author:** ![HelpComputer](https://avatars.discourse-cdn.com/v4/letter/h/ac8455/32.png) [@HelpComputer](https://discuss.elastic.co/u/HelpComputer)\ **Post date:** [October 8, 2020, 4:57pm UTC](https://discuss.elastic.co/t/assistance-with-my-logstash-filter/251342/8 "2020-10-08T16:57:23Z")
Hope this helps ```auto [2020-10-08T11:46:21,890][DEBUG][org.logstash.config.ir.PipelineConfig] -------- Logstash Config --------- [2020-10-08T11:46:21,896][DEBUG][org.logstash.config.ir.PipelineConfig] Config from source, source: LogStash::Config::Source::Local, pipeline_id:: main [2020-10-08T11:46:21,898][DEBUG][org.logstash.config.ir.PipelineConfig] Config string, protocol: file, id: /etc/logstash/conf.d/filter.conf [2020-10-08T11:46:21,899][DEBUG][org.logstash.config.ir.PipelineConfig] filter { if [log][file][path] == "/var/log/audit.log" { kv { include_keys => ["VlogRecNo", "VigilRecNo", "Pid", "TimeStamp", "Type", "Event", "LinuxPath", "netAddr_IPv4", "Dn", "UserDn", "PATH Type", "Comm"] } if [Comm] == "smdrd" { drop { } } } if ([type] == "o365_csv") { csv { columns => ["PSComputerName", "RunspaceId", "PSShowComputerName", "RecordType", "CreationDate", "UserIds", "Operations", "AuditData", "ResultIndex", "ResultCount", "Identity" "IsValid", "ObjectState"] skip_header => "true" if ([message] =~ /^#/) { drop { } } } date { match => ["CreationDate", "ISO8601"] } json { source => "AuditData" } } } ``` ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [October 8, 2020, 5:09pm UTC](https://discuss.elastic.co/t/assistance-with-my-logstash-filter/251342/9 "2020-10-08T17:09:28Z")
``` if ([message] =~ /^#/) { drop { } } ``` You cannot put that inside the csv {} filter. ---
**Author:** ![HelpComputer](https://avatars.discourse-cdn.com/v4/letter/h/ac8455/32.png) [@HelpComputer](https://discuss.elastic.co/u/HelpComputer)\ **Post date:** [October 9, 2020, 1:21pm UTC](https://discuss.elastic.co/t/assistance-with-my-logstash-filter/251342/10 "2020-10-09T13:21:03Z")
Thanks, that was it! Removed that part and it's now working. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 6, 2020, 1:21pm UTC](https://discuss.elastic.co/t/assistance-with-my-logstash-filter/251342/11 "2020-11-06T13:21:03Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.