Can't parse my data logs- -getting _grokparsefailure

Elastic Stack Logstash
1

Hello , any body can help me please to parse this log file (https_access files from apache tomcat) , i've try with many patterns but i always get ["_grokparsefailure"]

Those are two lines from my data logs :

172.23.10.200 46802 172.23.12.8 8084 172.23.10.200 POST - /PitneyBowse/Kering_Validate/MailingAddressPro 2208 2.208 2208 http-nio-8084-exec-36 - [05/Apr/2018:08:04:39 +0200] 200 704 - Apache-HttpClient/4.2.1 (java 1.5)

172.23.10.200 46732 172.23.12.8 8084 172.23.10.200 GET - ?q=organization.brand:"BV"+AND+name.last.local:(Koushik+OR+Koushik*)&&fl=client_id,name.last.local,num,phone.home.num,activity.registr_store /IODSTranscodes/diods1/customerv2 35 0.035 35 http-nio-8084-exec-36 - [05/Apr/2018:08:26:31 +0200] 200 11761 - Java/1.7.0_85

this is pattern that i use in grok :

grok {
match => { "message" =>"%{IP:address_ip_source} %{INT:port1} %{IP:address_ip_dest} %{INT:port2} %{IP:address_ip_source_dbl} %{WORD:request_method} - %{GREEDYDATA:message_or-request} %{INT:port3} %{NUMBER:duration} %{INT:port4} %{GREEDYDATA:http-nio} - %{TIMESTAMP_ISO8601:timestamp} %{INT:response_http_status_code} %{INT:nbr} %{GREEDYDATA:extra_msg} %{GREEDYDATA:extra_msg}"
}
}

thank you.

2

Hey Sara,

It looks like the timestamp you're trying to grok for is not a valid ISO 8601 format so it won't match the grok filter you're using:

TIMESTAMP_ISO8601 %{YEAR}-%{MONTHNUM}-%{MONTHDAY}[T ]%{HOUR}:?%{MINUTE}(?::?%{SECOND})?%{ISO8601_TIMEZONE}?

Perhaps try using \[%{HTTPDATE:timestamp}\] instead of {TIMESTAMP_ISO8601:timestamp}. It looks like it should parse correctly:

Data:
[05/Apr/2018:08:04:39 +0200]

Pattern:
\[%{HTTPDATE:timestamp}\]

Output:

{
  "timestamp": "05/Apr/2018:08:04:39 +0200"
}

I've tested on both your example data sets and it's parsing correctly, let me know if that works for you.

Cheers,
Mike