Logstash can not parse azure activity logs

Elastic Stack Logstash
1

Hello,
I have faced with issue during parsing Azure activity logs (multiline json) by Logstash.
Tried several codecs but all of them are useless. They can not parse multiline logs with coma delimiters.
Please advise how to fix it and add custom fields from Json file to the logstash index (logstash-*).
Thanks.

Json files sctucture is next:

	{
		 "time": "2017-02-28T12:04:25.7263914Z",
		 "resourceId": "/SUBSCRIPTIONS/...../RESOURCEGROUPS/....",
		 "operationName": "MICROSOFT.RESOURCES/SUBSCRIPTIONS/RESOURCEGROUPS/DELETE",
		 "category": "Delete",
		 "resultType": "Start",
		 "resultSignature": "Started.",
		 "durationMs": 0,
		 "callerIpAddress": "....",
		 "correlationId": "....",
		 "identity": {"authorization":{"scope":"....}},
		 "level": "Information",
		 "location": "global"
	}
	,
	{
		 "time": "2017-02-28T12:04:27.3201073Z",
		 "resourceId": "/SUBSCRIPTIONS/.../RESOURCEGROUPS/...",
		 "operationName": "MICROSOFT.RESOURCES/SUBSCRIPTIONS/RESOURCEGROUPS/DELETE",
		 "category": "Delete",
		 "resultType": "Accept",
		 "resultSignature": "Accepted.Accepted",
		 "durationMs": 1586,
		 "callerIpAddress": "....",
		 "correlationId": "....",
		 "identity": {"authorization":{"..."}},
		 "level": "Information",
		 "location": "global",
		 "properties": {"statusCode":"Accepted","serviceRequestId":null}
	}
	,
	ens so on.

Kibana shows _jsonparsefailure in tag field all the time.

2

Hello,
Found this topic. Best way to parse json input?

described there method works,

input {
exec {
command => "type c:\PT1H.json"
codec => json
interval => 60
}
}

but it reads only one entry from json and repeats this json record each time during the mentioned interval. (duplicates the same document in loop).

image.png1436×563 30.9 KB

Is there any method to read all entire json document from the begin till its end without time duplicates?

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.