Grok pars failure

Elastic Stack Logstash
1

hi dears,

i want to analyze log files like this:

""""""""
date=2018-11-07 time=12:49:48 bid=32679715 dvid=1085 itime=1541575641 log_id="20000018" type="attack" subtype="N/A" pri="None" msg_id=560216191 timezone="None" proto="tcp" service="https/tls1.2" src=192.168.1.50 src_port=559 dst=192.168.60.20 dst_port=80 policy="Net-Int" action="Alert_Deny" http_method="post" http_url="/SyncWeb/MobileSyncServlet" http_host="subdomain.domain.net" http_agent="IOS-5.4_23-iPhone-11.4" http_session_id="none" severity_level="Low" msg="Unauthorized Geo IP from Germany was not allowed" signature_subclass="N/A" signature_id="N/A" srccountry="Germany" content_switch_name="none" server_pool_name="Mobile-Pool" false_positive_mitigation="none" user_name="Unknown" monitor_status="Disabled" http_refer="none" http_version="1.x" dev_id="none" threat_weight="30" history_threat_weight="0" threat_level="High" main_type="GEO IP" device_id="FV-3KE3216000042" vd="root" devname="devWAF"

"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""'
And i wrote this grok for that:

grok {
match => { "message" => "^date=%{YEAR:year}-%{MONTHNUM:monthnum}-%{MONTHDAY:day} time=%{TIME:times} bid=%{INT:bid} dvid=%{INT:dvid} ititme=%{INT:itime} log_id="%{INT:logid}" type="%{WORD:attacktype}" subtype="%{WORD:subtype}" pri="%{WORD:priority}" msg_id=%{INT:msg_id} timezone="%{WORD:timezones}" proto=%{WORD:protocol} service="%{WORD:tlsprotocol}" src=%{IPORHOST:geo_src} src_port=%{POSINT:srcport} dst=%{IPORHOST:geo_dst} dst_port=%{POSINT:dstport} policy="%{DATA:policy}" action="%{WORD:wafaction}" http_method="%{WORD:httpmethod}" http_url="%{URIPATH:requri}" http_host="%{HOSTNAME:hostname}" http_agent="%{WORD:useragent}" http_session_id="%{DATA:session_id}" severity_level="%{WORD:level}" msg="%{DATA:waf_msg}" signature_subclass="%{DATA:signature_msg}" signature_id="%{DATA:signature_msg}" srccountry="%{WORD:country}" content_switch_name="%{DATA:content_switch}" server_pool_name="%{HOSTNAME:pool_name}" false_positive_mitigation="%{WORD:mitigation}" user_name="%{USERNAME:user}" monitor_status="%{WORD:monitorstatus}" http_refer="%{WORD:referrer}" http_version="%{NUMBER:httpversion}" dev_id="%{WORD:dev_id}" threat_weight="%{NUMBER:weight_treat}" history_threat_weight="%{NUMBER:history_treat}" threat_level="%{WORD:treate_level}" main_type="{DATA:main_type}" device_id="%{WORD:device_id}" vd="%{USERNAME:vd}" devname="%{WORD:devname}"$"}
}

i'm getting _grokparsfailure, it's very important to me and i don't know how can i grok this log. please give me some tips. Thank you

2

Use two windows. In one, run logstash with the -r option, so that it restarts every time the configuration is changed. In the other, run an editor and start with a configuration like

input { generator { count => 1 message => 'date=2018-11-07 time=12:49:48 bid=32679715 dvid=1085 itime=1541575641 log_id="20000018" type="attack" subtype="N/A" pri="None" msg_id=560216191 timezone="None" proto="tcp" service="https/tls1.2" src=192.168.1.50 src_port=559 dst=192.168.60.20 dst_port=80 policy="Net-Int" action="Alert_Deny" http_method="post" http_url="/SyncWeb/MobileSyncServlet" http_host="subdomain.domain.net" http_agent="IOS-5.4_23-iPhone-11.4" http_session_id="none" severity_level="Low" msg="Unauthorized Geo IP from Germany was not allowed" signature_subclass="N/A" signature_id="N/A" srccountry="Germany" content_switch_name="none" server_pool_name="Mobile-Pool" false_positive_mitigation="none" user_name="Unknown" monitor_status="Disabled" http_refer="none" http_version="1.x" dev_id="none" threat_weight="30" history_threat_weight="0" threat_level="High" main_type="GEO IP" device_id="FV-3KE3216000042" vd="root" devname="devWAF"' } }
filter {
    grok { match => { "message" => '^date=%{YEAR:year}-%{MONTHNUM:monthnum}-%{MONTHDAY:day} ' } }
}

output { stdout { codec => rubydebug { } } }

Then add one field at a time to the grok pattern and write the configuration out so that logstash re-reads it. Keep going until it breaks. Then fix the pattern that broke. The first place it breaks is 'ititme='.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.