O365 Module and o365.audit.ModifiedProperties.*.*

Elastic Stack Beats
1

I have the o365 module largely working great, however, one particular fields that needs further parsing is being problematic.
o365.audit.ModifiedProperties.*.*
Filebeat is not finishing the parsing of all the values in this field, leaving a singular o365.audit.ModifiedProperties field that looks like json, like:

{"o365":{
	"audit": {
		"ModifiedProperties": {
			"Action_Client_Name": {
				"NewValue": "DirectorySync"
			},
			"MethodExecutionResult_": {
				"NewValue": "Microsoft.Online.DirectoryServices.DirectoryUniquenessException"
			},
			"LastDirSyncTime": {
				"OldValue": "[\r\n \"2024-05-28T21:43:45Z\"\r\n]",
				"NewValue": "[\r\n \"2024-05-28T22:13:47Z\"\r\n]"
			},
			"Included_Updated_Properties": {
				"NewValue": "LastDirSyncTime"
			},
			"TargetId_UserType": {
				"NewValue": "Member"
			}
		}
	}
}}

I am trying to parse these values out to custom fields with logstash but nothing I have tried works.
I can start with a simple:

 json {
            source => ][o365][audit][ExtendedProperties][additionalDetails]"
            target => "[tl-custom][toParse][additionalDetails]"
        }

But I get constants warnings of
Error parsing json {:source=>"[o365][audit][ModifiedProperties]", :raw=>{}
because apparently "{}" is seen, which is either null or empty or the literal {} brackets, I'm not entirely sure.
So I started fronting that json input with

 if ![o365][audit][ModifiedProperties] or !([o365][audit][ModifiedProperties] =~ ".+") {
            mutate {
                remove_field => ["[o365][audit][ModifiedProperties]"] 
                add_tag => ["TEST"]    
            }
        }

which seems to resolve that warning issue. However, the logs seem to disappear when I do that, I never see ModifiedProperties field after that. Before the change, I see ModifiedProperties; After the change, I do not and I do not understand why. I also never see any events with the TEST tag so I know the mutate to remove the field was not hit.
Has anyone parsed these fields before?

1 Like