Error in config when logstash runs

[afs2r (Pierre)]

Hello,

First, logstash does not listen on tcp port 5544 as I mentioned in my config file...
And second, I cannot figure it out : I have this error repeatedly (the end of config file):

[2018-08-20T14:51:51,439][ERROR][logstash.agent ] Cannot create pipeline {:reason=>"Expected one of #, input, filter, output at line 35, column 1 (byte 1092) after "}

File /etc/logstash/conf.d/afs2rParser.conf

input {
beats { port => 5545 }
tcp { port => 5544 }
}

output {
elasticsearch { hosts => "localhost" }
}

filter {
grok {
patterns_dir => ["/etc/logstash/conf.d/patterns"]

match => { "message" => "@timestamp%{NONLETTER}%{TIMESTAMP_ISO8601:Date}%{GREEDYDATA}[%{DATA:Serveur}][%{GREEDYDATA}][%{LOGLEVEL:Severity}][%{DATA:Application}][%{DATA:Module}][%{DATA:Operation}][%{DATA:SubType}][%{GREEDYDATA:log}]"}

match => { "message" => "%{TIMESTAMP_ISO8601:Date} %{DATA:Serveur} %{GREEDYDATA:Application} %{POSINT} - [meta sequenceId="%{POSINT:sequenceId}"] %{GREEDYDATA:log}"} # Match a variety of any others

match => { "message" => "%{MONTH:month} %{MONTHDAY:monthday} %{TIME:time} %{DATA:Serveur} %{DATA:Application}: %{GREEDYDATA:log}"}

match => { "message" => "%{COMBINEDAPACHELOG}+%{GREEDYDATA:extra_fields}"}
}

geoip { source => "clientip" }

date {
match => ["Date", "ISO8601", "dd-MM-yy HH:mm:ss:SSS","dd/MMM/yyyy:HH:mm:ss Z"]
target => "@timestamp"
}

mutate {
rename => { "beat.hostname" => "Serveur" }
}
}

File /etc/logstash/patterns

PLAYDATE %{MONTHDAY}-%{MONTHNUM}-%{YEAR} %{TIME}
NONLETTER [,\:"]+

File /etc/logstash/logstash.yml

path.data: /var/lib/logstash
path.config: /etc/logstash/conf.d
path.logs: /var/log/logstash

logstash version : 5.6.10
Centos 6.10

Thanks a lot for your help !

Pierre

[magnusbaeck (Magnus Bäck)]

Do you have any other files in /etc/logstash/conf.d?

[afs2r (Pierre)]

No !

[_Alex (Alex Stiff)]

In your second match you may need to escape the quotes " depending on your intention:

[meta sequenceId=\"%{POSINT:sequenceId}\"]

[afs2r (Pierre)]

Ach the quotes are escaped, but backslash were removed in the post...

[_Alex (Alex Stiff)]

Okay! Another thing I can see is that you've defined patterns to be located in the conf.d directory:

patterns_dir => ["/etc/logstash/conf.d/patterns"]

But then listed the file as /etc/logstash/patterns. Maybe the inclusion of the patterns file inside the conf.d directory is causing problems (as @magnusbaeck eluded to). Moving the patterns file to live in /etc/logstash instead of the conf.d directory should resolve this.

[afs2r (Pierre)]

Yes seems good !
Thanks a lot !