Parsing nested JSON arrays

Elastic Stack Logstash
1

Hello,

I am trying to parse a message from Office 365 that contains a nested array with some values in each item. I'm not 100% sure how to tackle it, the array i'm working with looks like this:

"targetResources" => [
[0] {
           "displayName" => "Some oAuth App",
                  "type" => "ServicePrincipal",
    "modifiedProperties" => [
        [0] {
            "displayName" => "ConsentContext.IsAdminConsent",
               "oldValue" => nil,
               "newValue" => "\"False\""
        },
        [1] {
            "displayName" => "ConsentContext.IsAppOnly",
               "oldValue" => nil,
               "newValue" => "\"False\""
        },
        [2] {
            "displayName" => "ConsentContext.OnBehalfOfAll",
               "oldValue" => nil,
               "newValue" => "\"False\""
        },
        [3] {
            "displayName" => "ConsentContext.Tags",
               "oldValue" => nil,
               "newValue" => "\"WindowsAzureActiveDirectoryIntegratedApp\""
        },
        [5] {
            "displayName" => "TargetId.ServicePrincipalNames",
               "oldValue" => nil,
               "newValue" => "\"spn_aa\""
        }
    ],
                    "id" => "client_4b"
}
],

I want to be able to create filters and visualizations based on the data in [targetResources][modifiedProperties], like show all documents where ConsentContext.IsAdminConsent.newValue => true. What I think I would need to do is make the value of [targetResources][modifiedProperties][displayName] a field and nest old and new value underneath it like [targetResources][modifiedProperties][ConsentContext.IsAdminConsent][newValue] => false. Is this the correct way to go about it?

The other problem is that I can't seem to figure out how to accomplish it, I can access the values with mutate but in the event there are more than 6 items in the array i'm stuck. I've assumed i'm going to be doing this with ruby and have been reading through a lot of other peoples ruby code but i'm struggling with how the syntax works. I'm able to grab the values directly like the mutate but I can't seem to figure out how to loop through the array and do that to the old and new value objects.

filter {
  ruby {
    code => '
    parent = event.get("[json][modifiedProperties][0][displayName]")
    newkey = parent + ".newValue"
    oldkey = parent + ".oldValue"
    newValue = event.get("[json][modifiedProperties][0][newValue]")
    oldValue = event.get("[json][modifiedProperties][0][oldValue]")
    event.set(newkey, newValue)
    event.set(oldkey, oldValue)'
  }
}

Does anyone have any examples or suggestions on how to accomplish this?

2

Hi

You might try with the split filter.

Something like this:

filter {
  split {
    field => "[targetResources][modifiedProperties]"
  }
}

will give you one new event for each entry, regardless of how may there are.

Hope this helps

3

Take a look at this.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.