I am trying to parse a json file which consists of multiple line but even though I am using multiline codec but still I am getting error. Sample of my Json file is as follows:
> [
{
"action": "drop",
"clientASNDescription": "TRUE-AS-AP True Internet Co.,Ltd.",
"clientAsn": "17552",
"clientCountryName": "TH",
"clientIP": "xxx.xxx.xx.xx",
"clientRequestHTTPHost": "www.abc.com",
"clientRequestHTTPMethodName": "GET",
"clientRequestHTTPProtocol": "HTTP/1.1",
"clientRequestPath": "/xx/xx/xxx/xxx",
"clientRequestQuery": "",
"datetime": "2020-08-02T13:30:00Z",
"rayName": "6avbcghnasj",
"ruleId": "defcc1554sadsadsad",
"source": "firewallrules",
"userAgent": "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)",
"matchIndex": 0,
"metadata": [
{
"key": "filter",
"value": "asdasdsa2asd12a1s221asd"
},
{
"key": "type",
"value": "abcdef"
}
],
"sampleInterval": 1
}
]
My grok is as follow:
input {
file {
#type => "json"
path => "/home/kagamee/Downloads/jetairways_logs/*.json"
codec => multiline {pattern => '^\{' negate => true what => "previous" auto_flush_interval => 2 max_lines => 100000000 }
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
filter
{
mutate
{
replace => [ "message", "%{message}}" ]
gsub => [ 'message','\n','']
}
if [message] =~ /^{.*}$/
{
json { source => message }
}
}
output {
stdout { codec => rubydebug }
}
Thanks in advance