Grok matcher acting strange, works fine in grokconstructor

Elastic Stack Logstash
1

I'm using the below pattern to match these example logs:

2018-11-12 10:21:33,596[event-thread-1]INFO (HTTP-ENTRY)-ef8684ae-72a4-4ad7-94f9-603e7999b3a4 Get Request to             /session with method   POST from 10.134.236.205, 169.193.66.107 to  169.193.125.49:8008 return StatusCode 200 process in         34 millionSec client is myclient
2018-11-12 10:21:33,596[event-thread-1]INFO (HTTP-ENTRY)-ef8684ae-72a4-4ad7-94f9-603e7999b3a4 Get Request to             /session with method   POST from 10.111.236.205 to  169.123.234.49:8008 return StatusCode 200 process in         34 millionSec client is myclient


%{DATESTAMP:timestamp}\[%{DATA:thread}\]%{LOGLEVEL:logLevel}\s*\(%{DATA:program}\)\-%{UUID:id}\s*%{GREEDYDATA}Get Request to\s*%{URIPATHPARAM:request}\s*%{GREEDYDATA}with method\s*%{WORD:method}\s*%{GREEDYDATA}from\s*(?:(?:%{HOSTPORT:client_ip}|%{IP:client_ip})|((?:%{HOSTPORT:forwarding_ip}|%{IP:forwarding_ip}),\s*(?:%{HOSTPORT:client_ip}|%{IP:client_ip})))\s*to\s*(?:%{HOSTPORT:server_ip}|%{IP:server_ip})\s*%{GREEDYDATA}return StatusCode\s*%{NUMBER:statuscode}\s*%{GREEDYDATA}process in\s*%{NUMBER:duration}(?:%{GREEDYDATA}\s*is\s*%{USERNAME:client})?

In http://grokconstructor.appspot.com/do/match there is absolutely no problem.

However when there is a "forwarding ip" included in the logs (log line example #1, comma separated IPs) for some reason logstash puts a comma, space then a hyphen after SERVER_IP... E.G. -> "169.123.234.49:8008, - "

I have keep_empty_captures set to "true".

I cannot explain this behaviour. Any help?

EDIT - checking more logs coming in, it seems to be adding ", -" to IP fields everywhere. Is this something to do with keep_empty_captures maybe?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.