Grok type


(Vigneshprasanna Radhakrishnan) #1

what grok i should use for " (http-WEB0001VR%2F10.351.100.19-1000-15) "


(Magnus Bäck) #2

What is the desired result?


(Vigneshprasanna Radhakrishnan) #3

For below log the given grok is matching in grokdebug but not working in logstash any idea ??? :frowning:

Log:

2018-01-31 00:00:11,896 INFO [STDOUT] (http-WEB0001VR%2F10.351.100.19-1000-15) 00:00:11,896 [com.test.tst.connectivity.fddInvoker] INFO - TVS MSG Response [ CUSTOMER,PARTY.INFO/I/PROCESS///,***************//IN00111521,5269453 ]

Grok

%{TIMESTAMP_ISO8601:timestamp} *%{LOGLEVEL:Status} *[%{DATA:thread}] *(%{DATA:Server}%%{DATA:IP}-%{DATA:PORT}-%{DATA:Request_Response_ID} *%{TIME:Duration} *[%{JAVACLASS:Class}] *%{LOGLEVEL:Core_Status} - %{DATA:Response_From} *[%{GREEDYDATA:Log}]

Website Used : https://grokdebug.herokuapp.com/

!

The above image has the desired output

logstash.conf

Filter part :

filter {
grok {
match => [
"message",
"%{TIMESTAMP_ISO8601:timestamp} *%{LOGLEVEL:Status} *[%{DATA:thread}] *(%{DATA:Server}%%{DATA:IP}-%{DATA:PORT}-%{DATA:Request_Response_ID} *%{TIME:Duration} *[%{JAVACLASS:Class}] *%{LOGLEVEL:Core_Status} - %{DATA:Response_From} *[%{GREEDYDATA:Log}]"
]

}
}

Thanks in advance...


(Magnus Bäck) #4

Don't use more than one DATA or GREEDYDATA in the same expression. It's computationally expensive and unreliable. Addressing that might fix the problem you're having.


(Vigneshprasanna Radhakrishnan) #5

can you guide me with some other alternate grok ??


(Vigneshprasanna Radhakrishnan) #6

For (http-WEB0001VR%2F10.351.100.19-1000-15) i wanna split it into

"Server" : " http-WEB0001VR%2F10.351.100.19",

"Port" : "1000" &

"Response id" : "15"

How to do it without using DATA please advice.


(Muthu Pandi) #7

You can use below grok for splitting (http-WEB0001VR%2F10.351.100.19-1000-15)

(?[a-z]+-[a-zA-Z0-9]+%[a-zA-Z0-9]+.[0-9]+.[0-9]+.[0-9]+)-(?[0-9]+)-(?[0-9]+)

{
"Server": [
[
"http-WEB0001VR%2F10.351.100.19"
]
],
"port": [
[
"1000"
]
],
"Response id": [
[
"15"
]
]
}


(Vigneshprasanna Radhakrishnan) #8

Input Log:
2018-01-31 00:00:11,896 INFO [STDOUT] (http-WEB0001VR%2F10.351.100.19-1000-15) 00:00:11,896 [com.test.tst.connectivity.fddInvoker] INFO - TVS MSG Response [ CUSTOMER,PARTY.INFO/I/PROCESS///,***************//IN00111521,5269453 ]

Grok Used

%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:Status} *[%{DATA:thread}] (?[a-z]+-[a-zA-Z0-9]+%[a-zA-Z0-9]+.[0-9]+.[0-9]+.[0-9]+)-(?[0-9]+)-(?[0-9]+) *%{TIME:Duration} *[%{JAVACLASS:Class}] *%{LOGLEVEL:Core_Status} - %{DATA:Response_From} *[%{GREEDYDATA:Log}]

Separately its working but when put to gather its not working :frowning:


(Muthu Pandi) #9

I think you are missing something. Am able to get the filter working.


(system) #10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.