GROK parsing totally different log lines

[Vladpov]

Hey,

I have following log lines:

111.222.3.11 - - [15/Oct/2019:08:40:49 +0200] "asaa /ssss/asf sss/saaa" 200 12833 "-" "sada(asd; fffff sff ff f asdasd) ssssss/6sadasf (ddddd, sssssss) ddd/sdasds/2222"
222.333.3.11 - - [15/Oct/2019:08:41:01 +0200] "asd /asf.asf?afwa asf/2.0" 499 0 "-" "asf/5.0 (asf NT asf.asf; asf;asf x64; asf:asf.asf) asf/asf asfsaf/asf.0"
333.444.3.11 - - [15/Oct/2019:08:43:37 +0200] "ss /ssssddd/sssssss/agile/ssaaaaaaaaaaaaaddddddddddddddddddddddddffffffffffffffffffffffffffffffffffffffffffffffaaaaaaaaaaaaaaaaaaaaaaaasssssssssssssssssssssssssgggggggggggggggggggg 200 23333 "222222222222222222222222222222" "333asd/asf.af (asf; asf asf afs asf asf) asfasf/s (asf, asf asfasf) asfew/sssss"
555.555.111.5 - - [15/Oct/2019:08:44:45 +0200] "1111 /32132/222 HTTP/1.1" 403 152 "-" "Jajsaif (huasfd asdaefk)"

So every line is different and the same part is only IP in the beginning and date. What I need to do is to parse error code you can see in every line (499, 403, 200...). The problem is that the error part is always in different place in the line.

How can I filter it so I can see it for example in field Error: 409?

Thank you very much for help!

[Badger]

You could try

    grok { match => { "message" => [ "%{HTTPD_COMBINEDLOG}", "%{HTTPD_COMMONLOG}" ] } }

[Vladpov]

Impressive !

How does it actually work? Is it some kind of super patern made for error messages? Where can I find some documentation about it so I can understand it more...

Anyway, thank you for your help!

[Badger]

If you look in the directory ./vendor/bundle/jruby/2.5.0/gems/logstash-patterns-core-4.1.2/patterns under your logstash install directory (/usr/share/logstash) then there are a set of text files. These contain patterns that build upon the core grok patterns. The two I suggested are in the file httpd.

Underneath it all they are ruby regular expressions.

[Vladpov]

Thank you for further information!