# Splitting fields into new documents

**URL:** <https://discuss.elastic.co/t/splitting-fields-into-new-documents/251459>\
**Category:** Logstash\
**Created:** [October 8, 2020, 2:13pm UTC](https://discuss.elastic.co/t/splitting-fields-into-new-documents/251459 "2020-10-08T14:13:05Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![dudek1337](https://avatars.discourse-cdn.com/v4/letter/d/ccd318/32.png) [@dudek1337](https://discuss.elastic.co/u/dudek1337)\
**Post date:** [October 8, 2020, 2:13pm UTC](https://discuss.elastic.co/t/splitting-fields-into-new-documents/251459/1 "2020-10-08T14:13:05Z")

</div>

Hello

I have document like below

```auto
{"fsFreeSize.1":7801,"fsUsedRatio.3":23,"fsDevice.4":"dfbw_var","fsIndex.1":1,"fsUsedSize.3":130,"fsUsedRatio.5":1,"host":"hostname","fsDevice.5":"dfbw_dev#shm","fsIndex.4":4,"fsIndex.3":3,"fsUsedSize.1":2151,"fsFreeSize.5":15718,"@timestamp":"2020-10-08T13:57:01.320Z","fsIndex.2":2,"fsUsedSize.2":1876,"fsFreeSize.3":355,"fsUsedRatio.2":15,"fsFreeSize.2":8076,"fsIndex.5":5,"@version":"1","fsDevice.1":"dfbw_#","fsDevice.2":"dfbw_opt#denyall","fsUsedSize.5":0,"fsUsedRatio.4":1,"fsFreeSize.4":866103,"fsTotalSize.4":915581,"fsUsedRatio.1":18,"fsDevice.3":"dfbw_boot"} 

```

It is a output of SNMP walk plugin. I want to split each field that has .n into separate document

to illustrate, based on the image above I'd want it to look like:

1st document

```auto
fsIndex.1

fsFreeSize.1

fsUsedSize.1

fsUsedRatio.1

fsTotalSize.1

```

2nd document

```auto
fsIndex.2

fsFreeSize.2

fsUsedSize.2

fsUsedRatio.2

fsTotalSize.2

fsDevice.2

```

...

```auto
fsIndex.n

fsFreeSize.n

fsUsedSize.n

fsUsedRatio.n

fsTotalSize.n

fsDevice.n

```

Is something like this possible in Logstash? How would one go about creating this?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [October 8, 2020, 3:55pm UTC](https://discuss.elastic.co/t/splitting-fields-into-new-documents/251459/2 "2020-10-08T15:55:06Z")

</div>

You can do that using ruby

```
    json { source => "message" target => "[@metadata][json]" remove_field => ["message"] }
    ruby {
        code => '
            a = []
            a[0] = {}
            event.get("[@metadata][json]").each { |k, v|
                if k =~ /\w+\.\d+/
                    m = k.scan(/(\w+)\.(\d+)/)
                    a[m[0][1].to_i] ||= {}
                    a[m[0][1].to_i][m[0][0]] = v
                else
                    a[0][k] = v
                end
            }
            event.set("stuff", a)
        '
    }
    split { field => "stuff" }
```

---

<div class="post-metadata">

**Author:** ![dudek1337](https://avatars.discourse-cdn.com/v4/letter/d/ccd318/32.png) [@dudek1337](https://discuss.elastic.co/u/dudek1337)\
**Post date:** [October 12, 2020, 11:30am UTC](https://discuss.elastic.co/t/splitting-fields-into-new-documents/251459/3 "2020-10-12T11:30:54Z")

</div>

Thanks Badger for your solution! I didn't expect you to cook up something for me, but I really appreciate it.

Well I had to modify it to suit my need (Maybe there was easier way - but that's the only thing that I came up with)

I couldn't get the JSON thing to work, so I looped through event hash. Also when I split it I had all the fields from original event and I didn't want that so I remove them after they are processed.

Once again, thanks Badger - I learned a thing or two from you today!

Have a good one!

```auto
    ruby {
        code => '
            a = []
            a[0] = {}
            hash = event.to_hash
            hash.each { |k, v|
                if k =~ /\w+\.\d+/
                    m = k.scan(/(\w+)\.(\d+)/)
                    a[m[0][1].to_i] ||= {}
                    a[m[0][1].to_i][m[0][0]] = v
                    event.remove(k) if !k.start_with?("@", "storage", "host")
                    
                else
                    a[0][k] = v
                    event.remove(k) if !k.start_with?("@", "storage", "host")
                end
            }
            event.set("storage", a)
        '
    }
    split { field => "storage" }

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 9, 2020, 11:31am UTC](https://discuss.elastic.co/t/splitting-fields-into-new-documents/251459/4 "2020-11-09T11:31:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.