Split JSON into several events

Elastic Stack Logstash
1

Hi.
I'm trying to split into several events a JSON input.
I have something like this:

{
    "Domains": [
        {
            "name": "location",
            "hash_size": 4096,
            "AORs": [
                {
                    "AOR": "d1.580@my.domain.net",
                    "Contacts": [
                        {
                            "Contact": "sip:d1.580@16.9.246.120:4359;rinstance=d1.580-15b6daaca13bc79c4ff8da4c2ca8769d;transport=TLS",
                            "ContactID": "3173841468645438742",
                            "Expires": 105,
                            "Q": "",
                            "Callid": "ps0Y2-jLhWLY5oJMJqwZrw..@216.93.246.122",
                            "Cseq": 1163,
                            "User-agent": " Server 3.0.1 (22505)",
                            "State": "CS_SYNC",
                            "Flags": 0,
                            "Cflags": "",
                            "Socket": "tls:10.10.10.133:5061",
                            "Methods": 3567,
                            "SIP_instance": "<urn:uuid:e46b50f0-6e49-488a-b205-55f70df70718>"
                        }
                    ]
                },
                {
                    "AOR": "d1.558@my.domain.net",
                    "Contacts": [
                        {
                            "Contact": "sip:d1.558@192.168.6.4:5784;rinstance=f3b8df9d4d9b8285;transport=tls",
                            "ContactID": "4044167362323240131",
                            "Expires": 535,
                            "Q": "",
                            "Callid": "104875ZDI0NzE4MjdlOWRiZDNlYzY3NDhmMzM0MmVmODg4Zjc",
                            "Cseq": 43,
                            "User-agent": "SF 6.2.2.0 stamp 104875",
                            "Received": "sip:192.168.6.4:5784;transport=tls",
                            "State": "CS_SYNC",
                            "Flags": 0,
                            "Cflags": "SIP_PING NAT",
                            "Socket": "tls:10.10.10.133:5061",
                            "Methods": 5951
                        }
                    ]
                },
                {
                    "AOR": "d2.521@my.domain.net",
                    "Contacts": [
                        {
                            "Contact": "sip:d2.521@192.168.6.4:17422;rinstance=008dc2fffa48bde7;transport=tls",
                            "ContactID": "2339408900186963433",
                            "Expires": 351,
                            "Q": "",
                            "Callid": "104875NjQ4YzU5ZjA4YzQ1MTg1NzQ3OTgzMDEyMTQ1NTkxODg",
                            "Cseq": 582,
                            "User-agent": "SF 6.2.2.0 stamp 104875",
                            "Received": "sip:192.168.6.4:38431;transport=tls",
                            "State": "CS_SYNC",
                            "Flags": 0,
                            "Cflags": "SIP_PING NAT",
                            "Socket": "tls:10.10.10.133:5061",
                            "Methods": 5951
                         }
                    ]
                }
            ]
        }
    ]
}

What i need to do create an event per element in the AORs list, so in the example in need to convert that JSON array into three events like this:

{
                    "index": index1
                    "timestamp": XXXXXX
                    "AOR": "d2.521@my.domain.net",
                    "Contacts": [
                        {
                            "Contact": "sip:d2.521@192.168.6.4:17422;rinstance=008dc2fffa48bde7;transport=tls",
                            "ContactID": "2339408900186963433",
                            "Expires": 351,
                            "Q": "",
                            "Callid": "104875NjQ4YzU5ZjA4YzQ1MTg1NzQ3OTgzMDEyMTQ1NTkxODg",
                            "Cseq": 582,
                            "User-agent": "SF 6.2.2.0 stamp 104875",
                            "Received": "sip:192.168.6.4:38431;transport=tls",
                            "State": "CS_SYNC",
                            "Flags": 0,
                            "Cflags": "SIP_PING NAT",
                            "Socket": "tls:10.10.10.133:5061",
                            "Methods": 5951
                         }
}

This is my configuration...

input {
        exec {
          type => "register"
          command  => "/tmp/dump_users"
          codec => "json"
          interval => 30
        }
}

filter {

        json {
          source => "message"
        }

        split {
                field => "[message][Domains][AORs]"
        }

        dissect {
          mapping => {"AOR" => "%{anexo}@%{dominio}"}
        }
}

Now i'm getting erros like:

Only String and Array types are splittable. field:[message][Domains][AORs] is of type = NilClass
Jun 7 19:16:55 sbcar logstash: [2021-06-07T19:16:55,672][WARN ][org.logstash.dissect.Dissector][main][6860af3126033566880a5e3957363dfe4962009218f4a979c674a1725c4e1590] Dissector mapping, field not found in event

Hope someone can help me.
Thanks!

2

The field you are trying to split does not exist, so event.get is returning nil. The split filter does not check for that. Try

split { field => "[Domains][AORs]" }
3

Hi @Badger
Thanks for your reply...
I already try that.....

input {
        exec {
          type => "register"
          command  => "/tmp/dump_users"
          codec => "json"
          interval => 300
        }
}

filter {
        json {
          source => "message"
        }

        split {
                field => "[Domains][AORs]"
        }

        dissect {
          mapping => {"AOR" => "%{anexo}@%{dominio}"}
        }
}

But i keep getting this ERR and WARN:

Jun  8 10:35:27 sbcar logstash: [2021-06-08T10:35:27,727][WARN ][logstash.filters.split   ][main][11747525ca8be19897850de445f7282634b5fe59f97d11c1850daeef5897dc88] Only String and Array types are splittable. field:[Domains][AORs] is of type = NilClass
Jun  8 10:35:27 sbcar logstash: [2021-06-08T10:35:27,747][WARN ][org.logstash.dissect.Dissector][main][0daa284f3805b74149aa344bc98bf08f325122ce638256af1e00a927420eda4c] Dissector mapping, field not found in event

I was wondering if the syntaxt is the correct...
Thanks!

4

Your [Domains] field is also an array, so you would need to split on it if this has more than one element.

If the [Domains] is just an array with one element, you could use this to split:

split {
    field => "[Domains][0][AORS]"
}

You also need to change your dissect filter, it looks that the field AOR does not exist in the root level of the document, but it is a nested field inside AORs, so you need to use [AORs][AOR] instead of just [AOR].

5

Hi @leandrojmp
In fact!.. that made the trick!... now i have access to every AOR in the list.. as individual event!.
Thanks for your help!

Best Regards!

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.