# Matching log Using grok pattern

**URL:** <https://discuss.elastic.co/t/matching-log-using-grok-pattern/91578>\
**Category:** Logstash\
**Created:** [July 3, 2017, 1:07am UTC](https://discuss.elastic.co/t/matching-log-using-grok-pattern/91578 "2017-07-03T01:07:15Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![0314e7ff87c297426a9e](https://avatars.discourse-cdn.com/v4/letter/0/a3d4f5/32.png) [@0314e7ff87c297426a9e](https://discuss.elastic.co/u/0314e7ff87c297426a9e)\
**Post date:** [July 3, 2017, 1:07am UTC](https://discuss.elastic.co/t/matching-log-using-grok-pattern/91578/1 "2017-07-03T01:07:15Z")

</div>

i can't match log - field using grok patten in logstash

#Log

2015-06-27 01:32:00, 002 DEBUG [pool-2-thread-1 [container11-13]] db.QueryInterceptor (QueryInterceptor.java:57) - @@@@@@@@@ query @@@@@@@@@@  
SELECT A.CID  
, A.CDN\_CODE  
, B.FILE\_PATH  
,NVL2(ME.PATH,'/'||ME.PATH, B.FILE\_PATH||B.SAVE\_FILE\_NAME) AS SAVE\_FILE\_NAME  
, C.STAT  
,B.CONTENT\_NAME  
,B.PLAY\_TIME  
,B.BAND\_WIDTH  
,NVL2(ME.NSIZE,NSIZE, B.FILE\_SIZE) AS FILE\_SIZE  
,C.TRANS\_YN  
,NVL(C.SEND\_CNT , 0 ) AS SEND\_CNT  
,B.FILE\_TYPE  
,(SELECT  
EF.VIDEO\_BITRATE\_KBPS  
FROM  
MF\_M\_ENCODEPRESET EF  
, TB\_CDN\_PLATFORM\_MAP PM  
WHERE [EF.ID](http://EF.ID) = PM.ENCODEPRESET\_ID  
AND PM.COMPANY\_CODE = ?) AS TRANS\_BITRATE\_BYTES  
FROM TB\_CONTENT\_ASSET A  
, TB\_CONTENT B  
, TB\_CONTENT\_PUB\_STATUS C LEFT JOIN MF\_M\_ENCODEOUTPUT ME ON C.JOB\_ID = ME.JOB\_ID  
, TB\_CDN D  
WHERE 1=1  
AND A.CID = B.CID  
AND A.CID = C.CID  
AND A.CDN\_CODE = C.CDN\_CODE  
AND A.CDN\_CODE = D.COMPANY\_CODE  
AND A.CDN\_CODE = ?  
AND C.LINK\_TYPE = 'S'  
AND C.STAT IN ('W' , '22','E')  
AND C.SEND\_CNT \< 50  
ORDER BY C.STAT, C.UPDATE\_DATE, C.CID

#pattern ( logstash filtering )

match =\> { "message" =\> ["%{TIMESTAMP\_ISO8601:timestamp}, %{NUMBER:millis} %{LOGLEVEL:loglevel} [%{DATA:threads} [%{DATA:container}]] %{PACKAGE\_NAME:packageName} (%{SOURCE\_INFO}) - %{GREEDYDATA:logData}","%{TIMESTAMP\_ISO8601:timestamp}, %{NUMBER:millis} %{LOGLEVEL:loglevel} [%{DATA:threads} [%{DATA:container}]] %{PACKAGE\_NAME:packageName} (%{SOURCE\_INFO}) - %{GREEDYDATA:logData}%{SPACE}%{QUERY:query}"]  
}

#GROK PATTERN

PACKAGE\_NAME (?:[a-zA-Z0-9-]+.)+[A-Za-z0-9-]+  
SOURCE\_INFO (?:[A-Za-z0-9\_.-]+):%{NUMBER:line}

USERNAME [a-zA-Z0-9.\_-]+  
USER %{USERNAME}  
INT (?:[+-]?(?:[0-9]+))  
BASE10NUM (?\<![0-9.+-])(?\>[+-]?(?:(?:[0-9]+(?:.[0-9]+)?)|(?:.[0-9]+)))  
NUMBER (?:%{BASE10NUM})  
BASE16NUM (?\<![0-9A-Fa-f])(?:[+-]?(?:0x)?(?:[0-9A-Fa-f]+))  
BASE16FLOAT \b(?\<![0-9A-Fa-f.])(?:[+-]?(?:0x)?(?:(?:[0-9A-Fa-f]+(?:.[0-9A-Fa-f]\*)?)|(?:.[0-9A-Fa-f]+)))\b

POSINT \b(?:[1-9][0-9]_)\b  
WORD \b\w+\b  
NOTSPACE \S+  
SPACE \s_  
DATA .\*?

GREEDYDATA .\*  
QUOTEDSTRING (?\>(?\<!\)(?\>"(?\>\.|[^\"]+)+"|""|(?\>'(?\>\.|[^\']+)+')|''|(?\>`(?>\\.|[^\\`]+)+`)|``))  
UUID [A-Fa-f0-9]{8}-(?:[A-Fa-f0-9]{4}-){3}[A-Fa-f0-9]{12}

# URN, allowing use of RFC 2141 section 2.3 reserved characters

URN urn:[0-9A-Za-z][0-9A-Za-z-]{0,31}:(?:%[0-9a-fA-F]{2}|[0-9A-Za-z()+,.:=@;$\_!\*'/?#-])+

# paths

PATH (?:%{UNIXPATH}|%{WINPATH})  
UNIXPATH (/([\w\_%!$@:.,+~-]+|\.)\*)+  
TTY (?:/dev/(pts|tty([pq])?)(\w+)?/?(?:[0-9]+))

URIHOST %{IPORHOST}(?::%{POSINT:port})?

# uripath comes loosely from RFC1738, but mostly from what Firefox

# doesn't turn into %XX

URIPATH (?:/[A-Za-z0-9$.+!_'(){},~:;=@#%&\_-]_)+  
#URIPARAM ?(?:[A-Za-z0-9]+(?:=(?:[^&]_))?(?:&(?:[A-Za-z0-9]+(?:=(?:[^&]_))?)?)_)?  
URIPARAM ?[A-Za-z0-9$.+!_'|(){},~@#%&/=:;\_?-[]\<\>]\*  
URIPATHPARAM %{URIPATH}(?:%{URIPARAM})?  
URI %{URIPROTO}://(?:%{USER}(?::[^@]\*)?@)?(?:%{URIHOST})?(?:%{URIPATHPARAM})?

# Months: January, Feb, 3, 03, 12, December

MONTH \b(?:[Jj]an(?:uary|uar)?|[Ff]eb(?:ruary|ruar)?|Mm?r(?:ch|z)?|[Aa]pr(?:il)?|[Mm]a(?:y|i)?|[Jj]un(?:e|i)?|[Jj]ul(?:y)?|[Aa]ug(?:ust)?|[Ss]ep(?:tember)?|Oo?t(?:ober)?|[Nn]ov(?:ember)?|[Dd]e(?:c|z)(?:ember)?)\b  
MONTHNUM (?:0?[1-9]|1[0-2])  
MONTHNUM2 (?:0[1-9]|1[0-2])  
MONTHDAY (?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9])

# Days: Monday, Tue, Thu, etc...

DAY (?:Mon(?:day)?|Tue(?:sday)?|Wed(?:nesday)?|Thu(?:rsday)?|Fri(?:day)?|Sat(?:urday)?|Sun(?:day)?)

# Years?

YEAR (?\>\d\d){1,2}  
HOUR (?:2[0123]|[01]?[0-9])  
MINUTE (?:[0-5][0-9])

# '60' is a leap second in most time standards and thus is valid.

SECOND (?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?)  
TIME (?!\<[0-9])%{HOUR}:%{MINUTE}(?::%{SECOND})(?![0-9])

# datestamp is YYYY/MM/DD-HH:MM:SS.UUUU (or something like it)

DATE\_US %{MONTHNUM}[/-]%{MONTHDAY}[/-]%{YEAR}  
DATE\_EU %{MONTHDAY}[./-]%{MONTHNUM}[./-]%{YEAR}  
ISO8601\_TIMEZONE (?:Z|[+-]%{HOUR}(?::?%{MINUTE}))  
ISO8601\_SECOND (?:%{SECOND}|60)  
TIMESTAMP\_ISO8601 %{YEAR}-%{MONTHNUM}-%{MONTHDAY}[T]%{HOUR}:?%{MINUTE}(?::?%{SECOND})?%{ISO8601\_TIMEZONE}?  
DATE %{DATE\_US}|%{DATE\_EU}  
DATESTAMP %{DATE}[-]%{TIME}  
TZ (?:[APMCE][SD]T|UTC)  
DATESTAMP\_RFC822 %{DAY} %{MONTH} %{MONTHDAY} %{YEAR} %{TIME} %{TZ}  
DATESTAMP\_RFC2822 %{DAY}, %{MONTHDAY} %{MONTH} %{YEAR} %{TIME} %{ISO8601\_TIMEZONE}  
DATESTAMP\_OTHER %{DAY} %{MONTH} %{MONTHDAY} %{TIME} %{TZ} %{YEAR}  
DATESTAMP\_EVENTLOG %{YEAR}%{MONTHNUM2}%{MONTHDAY}%{HOUR}%{MINUTE}%{SECOND}

# Syslog Dates: Month Day HH:MM:SS

SYSLOGTIMESTAMP %{MONTH} +%{MONTHDAY} %{TIME}  
PROG [\x21-\x5a\x5c\x5e-\x7e]+  
SYSLOGPROG %{PROG:program}(?:[%{POSINT:pid}])?  
SYSLOGHOST %{IPORHOST}  
SYSLOGFACILITY \<%{NONNEGINT:facility}.%{NONNEGINT:priority}\>  
HTTPDATE %{MONTHDAY}/%{MONTH}/%{YEAR}:%{TIME} %{INT}

# Log formats

SYSLOGBASE %{SYSLOGTIMESTAMP:timestamp} (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource} %{SYSLOGPROG}:

# Log Levels

LOGLEVEL ([Aa]lert|ALERT|[Tt]race|TRACE|[Dd]ebug|DEBUG|[Nn]otice|NOTICE|[Ii]nfo|INFO|[Ww]arn?(?:ing)?|WARN?(?:ING)?|[Ee]rr?(?:or)?|ERR?(?:OR)?|[Cc]rit?(?:ical)?|CRIT?(?:ICAL)?|[Ff]atal|FATAL|[Ss]evere|SEVERE|EMERG(?:ENCY)?|[Ee]merg(?:ency)?)

#QUERY  
QUERY %{WORD:query\_type}(?:\n)?%{SPACE}(?:(?:[\a-zA-Z()'0-9-\_\*+,.]+?)?(?:\n)?%{SPACE}(?:FROM|INTO|TEMPORARY TABLE)(?:\n)?%{SPACE})?(?:%{WORD}.)?%{WORD:table}

SDATE (?:(\d\d){1,4}-\d{1,2}-(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9])\s(?:2[0123]|[01]?[0-9]):(?:[0-5][0-9]):(?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?))  
HEXID (%{BASE16NUM}:%{NUMBER})

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 3, 2017, 7:45am UTC](https://discuss.elastic.co/t/matching-log-using-grok-pattern/91578/2 "2017-07-03T07:45:38Z")

</div>

Did you have a question?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 31, 2017, 7:45am UTC](https://discuss.elastic.co/t/matching-log-using-grok-pattern/91578/3 "2017-07-31T07:45:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.