# Using logstash to denomalize data?

**URL:** <https://discuss.elastic.co/t/using-logstash-to-denomalize-data/294858>\
**Category:** Logstash\
**Created:** [January 19, 2022, 5:06pm UTC](https://discuss.elastic.co/t/using-logstash-to-denomalize-data/294858 "2022-01-19T17:06:25Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![dmarshall](https://avatars.discourse-cdn.com/v4/letter/d/77aa72/32.png) [@dmarshall](https://discuss.elastic.co/u/dmarshall)\
**Post date:** [January 19, 2022, 5:06pm UTC](https://discuss.elastic.co/t/using-logstash-to-denomalize-data/294858/1 "2022-01-19T17:06:26Z")

</div>

I have data coming in with field names like this:  
"field\_1" =\> "10"  
"field\_2" =\> "20"

and I'm trying to figure out how to convert it to something more like this:  
"field" =\> [{"num" =\> "1", "val" =\> "10"}, {"num" =\> "2", "val" =\> "20"}]

Issue is the source is sending in dozens of these fields, with an index that could theoretically go to 10k on each, so I'd end up with an index with potentially hundreds of thousands of fields.

I got tasked with this by my boss, but I really know next to nothing about logstash. For this, I'm not even sure what I'm looking for - just a function name would be useful so I could google it. :S

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 19, 2022, 6:16pm UTC](https://discuss.elastic.co/t/using-logstash-to-denomalize-data/294858/2 "2022-01-19T18:16:03Z")

</div>

Assuming that they are all top-level fields and the names really do start with "field\_" you could try something like

```
ruby {
    code => '
        fields = []
        event.to_hash.each { |k, v|
            if k =~ /^field_\d+/
                newK = k.sub(/^field_/, "")
                fields << { "num" => newK, "val" => v }
                event.remove(k)
            end
        if fields != []
            event.set("fields", fields)
        end
    '
}
```

---

<div class="post-metadata">

**Author:** ![dmarshall](https://avatars.discourse-cdn.com/v4/letter/d/77aa72/32.png) [@dmarshall](https://discuss.elastic.co/u/dmarshall)\
**Post date:** [January 19, 2022, 6:28pm UTC](https://discuss.elastic.co/t/using-logstash-to-denomalize-data/294858/3 "2022-01-19T18:28:43Z")

</div>

Thanks! I'll give this a shot.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 16, 2022, 6:29pm UTC](https://discuss.elastic.co/t/using-logstash-to-denomalize-data/294858/4 "2022-02-16T18:29:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.