# Unpack a list of hashes into new fields

**URL:** https://discuss.elastic.co/t/unpack-a-list-of-hashes-into-new-fields/290107
**Category:** Logstash
**Created:** [November 24, 2021, 9:39pm UTC](https://discuss.elastic.co/t/unpack-a-list-of-hashes-into-new-fields/290107 "2021-11-24T21:39:54Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![antonisnyc94](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/antonisnyc94/32/94316_2.png) [@antonisnyc94](https://discuss.elastic.co/u/antonisnyc94)
#### Post date: [November 24, 2021, 9:39pm UTC](https://discuss.elastic.co/t/unpack-a-list-of-hashes-into-new-fields/290107/1 "2021-11-24T21:39:54Z")

</div>

Hello,

I am having a field `data.aws.httpRequest.headers` , and it has a list of hashes as seen below.

```auto
{
  "name": "host",
  "value": "testwebsite.com"
},
{
  "name": "authorization",
  "value": "token hidden"
},
{
  "name": "sec-ch-ua",
  "value": "\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"96\", \"Google Chrome\";v=\"96\""
},
{
  "name": "sec-ch-ua-mobile",
  "value": "?0"
},
....

```

How can I use ruby to unpack the list into new fields, making the name of each hash in the list an additional field with its value as the value?

e.g `data.aws.httpRequest.headers.host: testwebsite.com` for the first one and so on?

I would greatly appreciate any help!

Thanks in advance,  
Tony

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [November 24, 2021, 11:25pm UTC](https://discuss.elastic.co/t/unpack-a-list-of-hashes-into-new-fields/290107/2 "2021-11-24T23:25:23Z")

</div>

[This](https://discuss.elastic.co/t/solved-split-filter-question-a-k-a-flatten-json-sub-array/130481/12) post should give you some ideas.

---

<div class="post-metadata">

### Author: ![antonisnyc94](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/antonisnyc94/32/94316_2.png) [@antonisnyc94](https://discuss.elastic.co/u/antonisnyc94)
#### Post date: [November 25, 2021, 1:59am UTC](https://discuss.elastic.co/t/unpack-a-list-of-hashes-into-new-fields/290107/3 "2021-11-25T01:59:26Z")

</div>

Thank you so much @Badger that indeed helped a lot! The only problem is that i want to keep the same name of the field and i cant figure out how to do this seamlessly through ruby. Do you know how to do it? The below code works but changing the field using logstash `mutate` seems kinda idiotic to me.

```auto
                  if [data][aws][httpRequest][headers] {
                    mutate {
                      rename => ["[data][aws][httpRequest][headers]", "[data][aws][httpRequest][headersTemp]"]
                    }
                    ruby {
                      code => '
                                data = event.get("[data][aws][httpRequest][headersTemp]")
                                data.each_index { |x|

                                    # remove token from authorization header
                                    if data[x]["name"] == "authorization" || data[x]["name"] == "Authorization" || data[x]["name"] == "X-Mandrill-Signature"
                                        data[x]["value"] = "token hidden"
                                    end

                                    #making the maps in the loop into their own field and capitilizing the name
                                    name = data[x]["name"].split(/ |\_|\-/).map(&:capitalize).join("-")
                                    value = data[x]["value"]

                                    event.set("[data][aws][httpRequest][headers][#{name}]", value)
                                }
                                '
                        }

                        mutate {
                        remove_field => ["[data][aws][httpRequest][headersTemp]" ]
                        }
                        
                      }

```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [November 25, 2021, 2:16am UTC](https://discuss.elastic.co/t/unpack-a-list-of-hashes-into-new-fields/290107/4 "2021-11-25T02:16:08Z")

</div>

In a ruby filter, event.remove deletes a field and also returns its value. So instead of renaming the field you could do

```
data = event.remove("[data][aws][httpRequest][headers]")
```

---

<div class="post-metadata">

### Author: ![antonisnyc94](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/antonisnyc94/32/94316_2.png) [@antonisnyc94](https://discuss.elastic.co/u/antonisnyc94)
#### Post date: [November 25, 2021, 2:39am UTC](https://discuss.elastic.co/t/unpack-a-list-of-hashes-into-new-fields/290107/5 "2021-11-25T02:39:02Z")

</div>

> [@Badger](#):
>
> `event.remove("[data][aws][httpRequest][headers]")`

Thank youu so much! It's wayyy simpler than expected!

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 23, 2021, 2:39am UTC](https://discuss.elastic.co/t/unpack-a-list-of-hashes-into-new-fields/290107/6 "2021-12-23T02:39:25Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
