# Howto Parse JSON into Hash

**URL:** <https://discuss.elastic.co/t/howto-parse-json-into-hash/84474>\
**Category:** Logstash\
**Created:** [May 4, 2017, 1:11am UTC](https://discuss.elastic.co/t/howto-parse-json-into-hash/84474 "2017-05-04T01:11:44Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![f9atanzh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/f9atanzh/32/17011_2.png) [@f9atanzh](https://discuss.elastic.co/u/f9atanzh)\
**Post date:** [May 4, 2017, 1:11am UTC](https://discuss.elastic.co/t/howto-parse-json-into-hash/84474/1 "2017-05-04T01:11:45Z")

</div>

Hello,

I'm working on reconfiguring log messages that come in as JSON strings so that I can POST them to an HTTP service with headers. Using beats input for Logstash. The main hurdle here is that I'd like to convert the "headers" part of the incoming JSON into a Hash that would be appropriate to send down to the http output plugin.

My `message` format (JSON):

```
{
	"headers": {
		"header0": "1.1",
		"header1": "",
		"header2": "307622"
	},
	"payload": "data"
}

```

My `configuration` (Logstash 5.3.0):

```
filter {
  json {
    source => ["message"]
  }

  mutate {
    add_field => {"headers" => "%{[headers]}"}
  }
}

```

This does convert the headers object into this:

```
logstash_1 | "headers" => [
logstash_1 | [0] [
logstash_1 | [0] "header0",
logstash_1 | [1] "1.1"
logstash_1 | ],
logstash_1 | [1] [
logstash_1 | [0] "header1",
logstash_1 | [1] nil
logstash_1 | ],
logstash_1 | [2] [
logstash_1 | [0] "header2",
logstash_1 | [1] "307622"
logstash_1 | ],
logstash_1 | [3] "{\"header1\":\"1.1\",\"header2\":null,\"header3\":\"307622\"}"
logstash_1 | ]

```

which is not quite there.

Has anybody here dealt with a similar problem? Could you recommend a alternative configuration?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 4, 2017, 5:34am UTC](https://discuss.elastic.co/t/howto-parse-json-into-hash/84474/2 "2017-05-04T05:34:48Z")

</div>

What does `message` contain?

> ```
> add_field => {"headers" => "%{[headers]}"}
> 
> ```

This doesn't make any sense, remove.

---

<div class="post-metadata">

**Author:** ![f9atanzh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/f9atanzh/32/17011_2.png) [@f9atanzh](https://discuss.elastic.co/u/f9atanzh)\
**Post date:** [May 4, 2017, 5:19pm UTC](https://discuss.elastic.co/t/howto-parse-json-into-hash/84474/3 "2017-05-04T17:19:50Z")

</div>

`Message` contains the following JSON:

```
{
	"headers": {
		"header0": "1.1",
		"header1": "",
		"header2": "307622"
	},
	"payload": "data"
}
```

---

<div class="post-metadata">

**Author:** ![f9atanzh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/f9atanzh/32/17011_2.png) [@f9atanzh](https://discuss.elastic.co/u/f9atanzh)\
**Post date:** [May 4, 2017, 10:21pm UTC](https://discuss.elastic.co/t/howto-parse-json-into-hash/84474/4 "2017-05-04T22:21:57Z")

</div>

Made some additional progress. Specifically, I started using the `ruby` filter plugin with the following configuration:

```
  ruby {
    init => "require 'json'"
    code => "
      headers_hash = JSON.parse(event.get('headers'))
      event.set('headers_hash', headers_hash)
    "
  }

```

This extracts all headers as JSON and puts them in a `Hashmap` "headers\_hash".

The issue that remains to be solved is that I can't correctly pass "headers\_hash" into the headers field for the http output plugin with the following configuration:

```
  http {
    http_method => "post"
    headers => headers_hash
    url => "http://localhost:5555/any-route"
    format => "message"
    message => [some_payload]
}

```

The error reported is:

```
 22:20:05.766 [LogStash::Runner] ERROR logstash.outputs.http - Invalid setting for http output plugin:

   output {
     http {
       # This setting must be a hash
       # This field must contain an even number of items, got 1
       headers => "headers_hash"
       ...
     }
   }

```

which is puzzling because "headers\_hash" is a `Hashmap`.

Any ideas?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 5, 2017, 5:11am UTC](https://discuss.elastic.co/t/howto-parse-json-into-hash/84474/5 "2017-05-05T05:11:17Z")

</div>

> This extracts all headers as JSON and puts them in a Hashmap "headers\_hash".

And that's exactly what the json filter does.

> which is puzzling because "headers\_hash" is a Hashmap.

Yes, but when the `headers` option is read it doesn't dereference its content, i.e.

```
   headers => "headers_hash"

```

means "the string 'headers\_hash'", not "the hash found in the field 'headers\_hash'".

I don't think it's possible to do what you want. Logstash's configuration language is too limited.

---

<div class="post-metadata">

**Author:** ![f9atanzh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/f9atanzh/32/17011_2.png) [@f9atanzh](https://discuss.elastic.co/u/f9atanzh)\
**Post date:** [May 5, 2017, 5:43pm UTC](https://discuss.elastic.co/t/howto-parse-json-into-hash/84474/6 "2017-05-05T17:43:56Z")

</div>

Yeah I dug in a bit deeper and confirmed that the `headers` option does not have the capability to interpolate the values passed into it.

Thank you for following this through.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 2, 2017, 5:46pm UTC](https://discuss.elastic.co/t/howto-parse-json-into-hash/84474/7 "2017-06-02T17:46:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.