I am using ELK 7.6.2
The problem is when I remove the kv plugin, there is log loaded to Elasticsearch, when I add kv plugin, there is zero data can be loaded to Elasticsearch.
Moreover, there is no error is logged in the logstash-plain.log
I have Googled but no solution is out there. Please help. Thank you.
The log sample is
Mar 24 16:51:40 2022 notice 111.2.333.44 111.22.333.44 time=16:51:39 devname="FG0000-NAT-B" devid="FG1K5Drfvv2345" logid="0000000013" type="traffic" subtype="forward" level="notice" vd="nat" eventtime=1648111900042439291 tz="+0800" srcip=11.222.333.144 srcport=523 srcintf="port34" srcintfrole="undefined" dstip=123.13.64.44 dstport=444 dstintf="port34" dstintfrole="undefined" sessionid=1355513311 proto=6 action="close" policyid=3 policytype="policy" poluuid="016d412dr0-72131298-51e7-3af0-d351asf32980" service="HTTPS" dstcountry="Singapore" srccountry="Reserved" trandisp="snat" transip=213.111.149.44 transport=2214 duration=2 sentbyte=4709 rcvdbyte=7217 sentpkt=23 rcvdpkt=17 shapingpolicyid=3 shaperperipname="ts-perip-web-dns" shaperperipdropbyte=0 appcat="unscanned"
In the filter plugin
grok {
match => { "message" => "^%{SYSLOGTIMESTAMP:timestamp}\s+%{YEAR:year}\s%{WORD:priority}\s%{IP:sourceIP}\s%{IP:host}\s%{GREEDYDATA:msg}$" }
}
kv {
source => "msg"
trim_value => "\""
value_split => "="
field_split_pattern => "\s+"
#remove_field => ["msg"]
}