# Json parsing
**URL:** https://discuss.elastic.co/t/json-parsing/105464
**Category:** Logstash
**Created:** [October 26, 2017, 5:58pm UTC](https://discuss.elastic.co/t/json-parsing/105464 "2017-10-26T17:58:29Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@Nikolay\_Petrov](https://discuss.elastic.co/u/Nikolay_Petrov)
#### Post date: [October 26, 2017, 5:58pm UTC](https://discuss.elastic.co/t/json-parsing/105464/1 "2017-10-26T17:58:29Z")
Hi folks,
could you please help to understand 'json' data processing eg.:
json data file:
_ssl.json:_
> [
> {
> "certname": "host1",
> "environment": "nk",
> "name": "ruby",
> "value": {
> "platform": "x86\_64-linux",
> "sitedir": "/path/site",
> "version": "2.1.9"
> }
> },
> {
> "certname": "host2",
> "environment": "staging",
> "name": "memorysize\_mb",
> "value": 7986.953125
> }
> ]
I'm trying to store the new incoming json data(see above) in separate index with fields namae grabbing by json, but my fault most likely is not enough experience on this matter and I have as a result that all data is stored in one big message field like:
To get it, i setup a http server:
> input {
> http {
> host =\> "192.168.1.2"
> port =\> 8088
> }
> }
> I'm not sure that message are going through this filter because (see below how data is posting to elastic):
> filter {
> if [headers][request\_path] == "/ssl-events" {
> json {
> source =\> "message"
> }
> }
> mutate {
> replace =\> ["message", "%{message}"]
> gsub =\> ['message','\n','']
> }
> if [message] =~ /^{.\*}$/ {
> json { source =\> message }
> }
> }
> output {
> if [headers][request\_path] == "/ssl-events" {
> elasticsearch {
> codec =\> json
> hosts =\> "192.168.1.2:9200"
> manage\_template =\> false
> index =\> "ssl-events-%{+YYYY.MM.dd}"
> document\_type =\> "%{[@metadata][type]}"
> }
> }
> }
posting data to elastic
> curl -XPOST [http://192.168.1.2:8088/ssl-events](http://192.168.1.2:8088/ssl-events) -d '@ssl.json'
The question is: how to setup in the configuration to be able to store the data in elastic field like in json file:
[item]: [value] :
> "certname": "host1"
> "environment": "nk"
> "name": "ruby"
> "platform": "x86\_64-linux"
> "sitedir": "/path/site"
> "version": "2.1.9"
> "certname": "host2"
> "environment": "staging"
> "name": "memorysize\_mb"
> "value": 7986.953125
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [October 27, 2017, 5:06am UTC](https://discuss.elastic.co/t/json-parsing/105464/2 "2017-10-27T05:06:21Z")
Your configuration looks reasonable. For now, replace the elasticsearch output with a `stdout { codec => rubydebug }` output. What do you get in the Logstash log when you post ssl.json to Logstash?
> replace =\> ["message", "%{message}"]
This doesn't make any sense.
> codec =\> json
Remove.
---
### Author:  [@Nikolay\_Petrov](https://discuss.elastic.co/u/Nikolay_Petrov)
#### Post date: [October 27, 2017, 7:38am UTC](https://discuss.elastic.co/t/json-parsing/105464/3 "2017-10-27T07:38:25Z")
Thanks for your reply!
I was change configuration based on your recommendations.
My json file has ~3million lines(the example from above was just in short form).
When i post it to the elastic logstash crashed with:
> [2017-10-27T07:29:02,883][FATAL][logstash.runner] An unexpected error occurred! {:error=\>#\>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/1.9/gems/awesome_print-1.8.0/lib/awesome_print/inspector.rb:163:in `merge\_custom\_defaults!'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/awesome\_print-1.8.0/lib/awesome\_print/inspector.rb:50:in `initialize'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/awesome_print-1.8.0/lib/awesome_print/core_ext/kernel.rb:9:in `ai'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-codec-rubydebug-3.0.4/lib/logstash/codecs/rubydebug.rb:39:in `encode_default'", "org/jruby/RubyMethod.java:120:in `call'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-codec-rubydebug-3.0.4/lib/logstash/codecs/rubydebug.rb:35:in `encode'", "/usr/share/logstash/logstash-core/lib/logstash/codecs/base.rb:50:in `multi\_encode'", "org/jruby/RubyArray.java:1613:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/codecs/base.rb:50:in `multi\_encode'", "/usr/share/logstash/logstash-core/lib/logstash/outputs/base.rb:90:in `multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/output_delegator_strategies/single.rb:15:in `multi\_receive'", "org/jruby/ext/thread/Mutex.java:149:in `synchronize'", "/usr/share/logstash/logstash-core/lib/logstash/output_delegator_strategies/single.rb:14:in `multi\_receive'", "/usr/share/logstash/logstash-core/lib/logstash/output\_delegator.rb:49:in `multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:436:in `output\_batch'", "org/jruby/RubyHash.java:1342:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:435:in `output\_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:381:in `worker_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:342:in `start\_workers'"]}
and I'm getting expected behaviour from curl that server is down:
> curl: (52) Empty reply from server
and after that logstash is auto restarting by initctl and return to normal state.
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [October 27, 2017, 8:45am UTC](https://discuss.elastic.co/t/json-parsing/105464/4 "2017-10-27T08:45:19Z")
Appears to be a known bug ([https://github.com/logstash-plugins/logstash-output-stdout/issues/11](https://github.com/logstash-plugins/logstash-output-stdout/issues/11)). You can use a json\_lines codec instead.
---
### Author:  [@Nikolay\_Petrov](https://discuss.elastic.co/u/Nikolay_Petrov)
#### Post date: [November 13, 2017, 9:14pm UTC](https://discuss.elastic.co/t/json-parsing/105464/5 "2017-11-13T21:14:10Z")
Fixed for now.
Thanks a lot.
Let mark this case fixed with suggested solution(it works if disable codec rubydebug).
Just in case I will re-open or rise another one
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 11, 2017, 9:14pm UTC](https://discuss.elastic.co/t/json-parsing/105464/6 "2017-12-11T21:14:24Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.