Splitting fields into new documents

[dudek1337]

Hello

I have document like below

{"fsFreeSize.1":7801,"fsUsedRatio.3":23,"fsDevice.4":"dfbw_var","fsIndex.1":1,"fsUsedSize.3":130,"fsUsedRatio.5":1,"host":"hostname","fsDevice.5":"dfbw_dev#shm","fsIndex.4":4,"fsIndex.3":3,"fsUsedSize.1":2151,"fsFreeSize.5":15718,"@timestamp":"2020-10-08T13:57:01.320Z","fsIndex.2":2,"fsUsedSize.2":1876,"fsFreeSize.3":355,"fsUsedRatio.2":15,"fsFreeSize.2":8076,"fsIndex.5":5,"@version":"1","fsDevice.1":"dfbw_#","fsDevice.2":"dfbw_opt#denyall","fsUsedSize.5":0,"fsUsedRatio.4":1,"fsFreeSize.4":866103,"fsTotalSize.4":915581,"fsUsedRatio.1":18,"fsDevice.3":"dfbw_boot"} 

It is a output of SNMP walk plugin. I want to split each field that has .n into separate document

to illustrate, based on the image above I'd want it to look like:

1st document

fsIndex.1

fsFreeSize.1

fsUsedSize.1

fsUsedRatio.1

fsTotalSize.1

2nd document

fsIndex.2

fsFreeSize.2

fsUsedSize.2

fsUsedRatio.2

fsTotalSize.2

fsDevice.2

...

fsIndex.n

fsFreeSize.n

fsUsedSize.n

fsUsedRatio.n

fsTotalSize.n

fsDevice.n

Is something like this possible in Logstash? How would one go about creating this?

[Badger]

You can do that using ruby

    json { source => "message" target => "[@metadata][json]" remove_field => [ "message" ] }
    ruby {
        code => '
            a = []
            a[0] = {}
            event.get("[@metadata][json]").each { |k, v|
                if k =~ /\w+\.\d+/
                    m = k.scan(/(\w+)\.(\d+)/)
                    a[m[0][1].to_i] ||= {}
                    a[m[0][1].to_i][m[0][0]] = v
                else
                    a[0][k] = v
                end
            }
            event.set("stuff", a)
        '
    }
    split { field => "stuff" }

[dudek1337]

Thanks Badger for your solution! I didn't expect you to cook up something for me, but I really appreciate it.

Well I had to modify it to suit my need (Maybe there was easier way - but that's the only thing that I came up with)

I couldn't get the JSON thing to work, so I looped through event hash. Also when I split it I had all the fields from original event and I didn't want that so I remove them after they are processed.

Once again, thanks Badger - I learned a thing or two from you today!

Have a good one!

    ruby {
        code => '
            a = []
            a[0] = {}
            hash = event.to_hash
            hash.each { |k, v|
                if k =~ /\w+\.\d+/
                    m = k.scan(/(\w+)\.(\d+)/)
                    a[m[0][1].to_i] ||= {}
                    a[m[0][1].to_i][m[0][0]] = v
                    event.remove(k) if !k.start_with?("@", "storage", "host")
                    
                else
                    a[0][k] = v
                    event.remove(k) if !k.start_with?("@", "storage", "host")
                end
            }
            event.set("storage", a)
        '
    }
    split { field => "storage" }