# JSON parsing problem

**URL:** <https://discuss.elastic.co/t/json-parsing-problem/166413>\
**Category:** Logstash\
**Created:** [January 30, 2019, 6:29pm UTC](https://discuss.elastic.co/t/json-parsing-problem/166413 "2019-01-30T18:29:14Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Arthur\_Bulakaiev](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arthur_bulakaiev/32/20893_2.png) [@Arthur\_Bulakaiev](https://discuss.elastic.co/u/Arthur_Bulakaiev)\
**Post date:** [January 30, 2019, 6:29pm UTC](https://discuss.elastic.co/t/json-parsing-problem/166413/1 "2019-01-30T18:29:14Z")

</div>

Hi there!

My target is very simple (as it seems): I want logstash to receive a json from http, process this json with some scrypt, and bypass it as json next.

I've started with json codec, but I've not found **any** way to properly access the root of event object, so I could not get the whole json content. So, I saw this [How to read JSON input sent to Http input plugin in filter section](https://discuss.elastic.co/t/how-to-read-json-input-sent-to-http-input-plugin-in-filter-section/118090) topic and tried to make config like in the answer.

Let's say I've this config now:

```
input {
  http {
    id => "my_plugin_id"
	port => 12345
	additional_codecs => { }
  }
}

filter {
	json {
		source => "message"
		target => "json"
    }
#one day here will be the ruby scrypt
} 

```

and this json:  
{  
"some":  
{  
"some2":"SOME-AUTOGENERATED-ID1",  
"some3":"stat"  
},  
"user\_name":"some4",  
"machine\_name":"SOME",  
"install\_ver":"SOME",  
"type":"SOME",  
"message\_desc":"test",  
"time":"2002-02-10T16:58:48.000+0200",  
"win\_ver":"Some version",  
"proc\_ver":"version"  
}

And what I'm trying to do, is using curl to send my json on this port:  
`curl -H "Content-Type: application/json" -XPOST "localhost:12345" --data-binary @stat2.json`

Instead of having the real json in the json field of output, I really have this from logstash:  
"json" =\> {  
"install\_ver" =\> "SOME",  
"user\_name" =\> "some4",  
"machine\_name" =\> "SOME",  
"time" =\> "2002-02-10T16:58:48.000+0200",  
"win\_ver" =\> "Windows 10 1703",  
"type" =\> "SOME",  
"some" =\> {  
"some2" =\> "SOME-AUTOGENERATED-ID1",  
"some3" =\> "stat"  
},  
With =\> separator between key and value instead of colon. It's definetely _not_ a json.

I've already tried to do  
mutate { gsub =\> ["json", "=\>", ':'] }  
but it makes no action with json field, only the text or arrays could be here.

I just want to have real json in the "json" field, with colon separators. Of cource I can replace it with the scrypt, but it seems like very common thing, and I'm looking for the native solution.

Could someone here help me, please?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 30, 2019, 6:46pm UTC](https://discuss.elastic.co/t/json-parsing-problem/166413/2 "2019-01-30T18:46:59Z")

</div>

What output are you using? Have you specified a codec on it?

---

<div class="post-metadata">

**Author:** ![Arthur\_Bulakaiev](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arthur_bulakaiev/32/20893_2.png) [@Arthur\_Bulakaiev](https://discuss.elastic.co/u/Arthur_Bulakaiev)\
**Post date:** [January 30, 2019, 6:54pm UTC](https://discuss.elastic.co/t/json-parsing-problem/166413/3 "2019-01-30T18:54:31Z")

</div>

I am using such output:

```
output {
  stdout { 
        codec => rubydebug 
    } 
}

```

I've also tried to use just stdout, without a codec.

But I beleive it's not an output setting issue, because if I try to do smth like this in filter:  
`ruby {code => 'open("test.json", "w") { |file| file.write(event.get("json")) } }`  
in the test.json file I'll have same =\> stuff.

And if my information is correct, the pipeline is input -\> filter -\> output.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 30, 2019, 6:56pm UTC](https://discuss.elastic.co/t/json-parsing-problem/166413/4 "2019-01-30T18:56:43Z")

</div>

The format with "fieldname" =\> "fieldvalue" is what rubydebug does. It is working as expected. If you want json then just tell it that.

```
output { stdout { codec => json_lines } }
```

---

<div class="post-metadata">

**Author:** ![Arthur\_Bulakaiev](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arthur_bulakaiev/32/20893_2.png) [@Arthur\_Bulakaiev](https://discuss.elastic.co/u/Arthur_Bulakaiev)\
**Post date:** [January 30, 2019, 7:02pm UTC](https://discuss.elastic.co/t/json-parsing-problem/166413/5 "2019-01-30T19:02:34Z")

</div>

It really helps, but just for the stdout. Now I've unformatted correct json in the console output, but still =\> stuff in the file I created through ruby, on filter stage.

Is there such method to fix the separator on filter stage?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 30, 2019, 7:17pm UTC](https://discuss.elastic.co/t/json-parsing-problem/166413/6 "2019-01-30T19:17:42Z")

</div>

You are using a ruby filter to write to a file? Can you show us what that filter looks like? It will probably end up being the same problem in another guise.

---

<div class="post-metadata">

**Author:** ![Arthur\_Bulakaiev](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arthur_bulakaiev/32/20893_2.png) [@Arthur\_Bulakaiev](https://discuss.elastic.co/u/Arthur_Bulakaiev)\
**Post date:** [January 31, 2019, 9:46am UTC](https://discuss.elastic.co/t/json-parsing-problem/166413/7 "2019-01-31T09:46:12Z")

</div>

Yes.  
The filter code looks like this:  
filter {  
json {  
source =\> "message"  
target =\> "json"  
}  
ruby {code ='open("test.json", "w") { |file| file.write(event.get("json")) }' }  
}

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 31, 2019, 1:59pm UTC](https://discuss.elastic.co/t/json-parsing-problem/166413/8 "2019-01-31T13:59:11Z")

</div>

> [@Arthur\_Bulakaiev](#):
>
> ruby {code ='open("test.json", "w") { |file| file.write(event.get("json")) }' }

```
ruby {code => 'open("/tmp/test.json", "w") { |file| file.write(event.get("json").to_json) }' }

```

Note the .to\_json at the end.

---

<div class="post-metadata">

**Author:** ![Arthur\_Bulakaiev](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arthur_bulakaiev/32/20893_2.png) [@Arthur\_Bulakaiev](https://discuss.elastic.co/u/Arthur_Bulakaiev)\
**Post date:** [January 31, 2019, 2:51pm UTC](https://discuss.elastic.co/t/json-parsing-problem/166413/9 "2019-01-31T14:51:31Z")

</div>

.to\_json works perfectly, thank you Badger!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 28, 2019, 2:51pm UTC](https://discuss.elastic.co/t/json-parsing-problem/166413/10 "2019-02-28T14:51:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.