Lostash grok parse failure

[Rajat_Singh]

Hi folks,
I had created a logstash pattern to parse the file, when i validating it grok debugger in first it is not working but when i am removing space and then again creating space, then it is working

10.150.86.92 - - [31/May/2017:13:41:03 +0530] "GET / HTTP/1.1" 200 14123 "https://perf.anntaylor.com/international/contextChooser.jsp" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36" "-" 18 18920055

%{IPV4:clientip} - - [(?%{MONTHDAY:monthday}/%{MONTH:monthname}/%{YEAR:year}:%{HOUR:hour}:%{MINUTE:minute}:%{SECOND:second} %{ISO8601_TIMEZONE:timezone}]) "%{WORD:method} %{NOTSPACE:urlpath} %{NOTSPACE:protocolversion}" %{INT:httpresponse} %{INT:bytes} "%{URI:referer}" "%{GREEDYDATA:useragent}" "%{NOTSPACE:transactionname}" %{INT:responsetime_sec} %{INT:responsetime_micro}

at 18 in logs i am removing two space first and then again i am again creating two space it is working fine.

I am not able to identify the issue .

[paz]

Your posted grok pattern seems to be failing to compile on grokdebugger, maybe you have mis-pasted it?

Anyhow the below should work (escaped the square brackets and removed the (? part).

Btw, you should consider replacing the URI pattern for the referrer field with a simple NOTSPACE one, if you a) know you are going to get legitimate links in the logs, or b) don't care about URI validation via pattern failures since it's quite more lightweight.

[Rajat_Singh]

10.150.86.92 - - [31/May/2017:13:44:46 +0530] "POST /cws/findinstore/findInStoreResults.jsp HTTP/1.1" 200 466 "https://perf.anntaylor.com/cricket-sweater/430164?skuId=22440747&defaultColor=2089&colorExplode=false&catid=cata000011" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36" "-" 0 473549

its not working for this log which is same as the above log

Thanks for the reply

[paz]

Hmm, weird. Both log lines work on the grokdebug for me with the pattern I posted.

Are you trying it on grokdebug or in Logstash?
If in Logstash, is there a _grokfailure tag, or errors occuring by the rest of the configuration, if any?
Can you identify the offending patterns? (the respective fields should be missing from the resulting document)

[Rajat_Singh]

In grok debugger it working fine but in logstash it is getting failed .

no errors i am getting on it .

[paz]

That is odd. I tried it both on LS 2.4 and 5.x (both Windows and Linux) and it works as intended.

I'd try starting with just one pattern, e.g. clientip only and see how it behaves (there should be no reason for it to fail, unless there's some weird character encoding), and work my way up from there.

[Rajat_Singh]

Thanks for the help .
I had found workaround for it copied the space from the logs itself in my pattern it is was working fine,
I think issue was that some carriage character was their in the logs.