# Logstash parsing json

**URL:** <https://discuss.elastic.co/t/logstash-parsing-json/265163>\
**Category:** Logstash\
**Created:** [February 23, 2021, 6:37am UTC](https://discuss.elastic.co/t/logstash-parsing-json/265163 "2021-02-23T06:37:09Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![ali\_ghorbani](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ali_ghorbani/32/84372_2.png) [@ali\_ghorbani](https://discuss.elastic.co/u/ali_ghorbani)\
**Post date:** [February 23, 2021, 6:37am UTC](https://discuss.elastic.co/t/logstash-parsing-json/265163/1 "2021-02-23T06:37:09Z")

</div>

I am trying to use log-stash to read input file 1.log in JSON format and write on elasticsearch. This is my log file:

```auto
{"key":"value00"}
{"key":"value01"}
{"key1":[{"key2":"value02"},{"key3":"value03"},{"key4":[{"key5":"value 04"}]}]}

```

and this is my configuration file:

```auto
input {
  file {
    type => "json"
    path => "/logstash/1.log"
  }
}
filter{
  json {
    source => "message"
    remove_field => ["message"]
  }
}
output {
    elasticsearch {
        hosts => ["192.168.1.6:9200"]
        user => "elastic"
        password => "something"
    }
}

```

the log-stash behaviour is completely random. Some times it works correctly but, some times it returns the following error for the same input structure:

```auto
Error parsing json {:source=>"message", :raw=>"4\"}]}]}", :exception=>#<LogStash::Json::ParserError: Unexpected character ('"' (code 34)): Expected space separating root-level values

```

---

<div class="post-metadata">

**Author:** ![grumo35](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grumo35/32/59451_2.png) [@grumo35](https://discuss.elastic.co/u/grumo35)\
**Post date:** [February 23, 2021, 8:40am UTC](https://discuss.elastic.co/t/logstash-parsing-json/265163/2 "2021-02-23T08:40:29Z")

</div>

Hi,

Which version of logstash are you running ?

The type field is deprecated you should get rid of it in the input.

Also, can you provide sample of your json formatted data ?

---

<div class="post-metadata">

**Author:** ![ali\_ghorbani](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ali_ghorbani/32/84372_2.png) [@ali\_ghorbani](https://discuss.elastic.co/u/ali_ghorbani)\
**Post date:** [February 23, 2021, 10:14am UTC](https://discuss.elastic.co/t/logstash-parsing-json/265163/3 "2021-02-23T10:14:48Z")

</div>

Hi,  
thanks for your response. I'm using logstash version 7.11.0.  
Also removing type filed cause following warnings and the out put in elastic is like the figure.

```auto
[WARN] 2021-02-23 13:32:08.679 [[main]>worker2] json - Error parsing json {:source=>"message", :raw=>"22\":\"value01\"}", :exception=>#<LogStash::Json::ParserError: Unexpected character ('"' (code 34)): Expected space separating root-level values
 at [Source: (byte[])"22":"value01"}"; line: 1, column: 4]>}
[WARN] 2021-02-23 13:32:08.703 [[main]>worker1] elasticsearch - Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"logstash", :routing=>nil, :_type=>"_doc"}, #<LogStash::Event:0x433f6176>], :response=>{"index"=>{"_index"=>"logstash-2021.02.16-000001", "_type"=>"_doc", "_id"=>"4wdWzncBWvA1N8Dsw_u8", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [key1] of type [text] in document with id '4wdWzncBWvA1N8Dsw_u8'. Preview of field's value: '{key2=value02}'", "caused_by"=>{"type"=>"illegal_state_exception", "reason"=>"Can't get text on a START_OBJECT at 1:64"}}}}}

```

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/f/0f9c59d7abb08ee8025b36833e46db30a9865c48.png)

if you mean the correct out put in elastic by json formatted data its like this for something like last json in input data:

```auto
{
  "_index": "logstash-2021.02.16-000001",
  "_type": "_doc",
  "_id": "Ccx7yHcBz9PWCXgpyOE8",
  "_version": 1,
  "_score": null,
  "_source": {
    "ali222": [
      {
        "ere1231za": "123123michel"
      },
      {
        "ahm123123ad": "ali123123"
      },
      {
        "reza": [
          {
            "ali": "asdas22d"
          }
        ]
      }
    ],
    "host": "blue",
    "type": "json",
    "path": "/home/ali/logstash/1.log",
    "@version": "1",
    "@timestamp": "2021-02-22T06:44:51.259Z"
  },
  "fields": {
    "@timestamp": [
      "2021-02-22T06:44:51.259Z"
    ]
  },
  "sort": [
    1613976291259
  ]
}

```

I'm really appreciate your help.

---

<div class="post-metadata">

**Author:** ![grumo35](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grumo35/32/59451_2.png) [@grumo35](https://discuss.elastic.co/u/grumo35)\
**Post date:** [February 23, 2021, 10:26am UTC](https://discuss.elastic.co/t/logstash-parsing-json/265163/4 "2021-02-23T10:26:04Z")

</div>

> [@ali\_ghorbani](#):
>
> `reason"=>"Can't get text on a START_OBJECT at 1:64"}}}}`

> [@ali\_ghorbani](#):
>
> if you mean the correct out put in elastic by json formatted data its like this for something like last json in input data:

I meant the input data contained in the file 1.log i'm affraid it's not valid json data i was asking for a log line of this file.

As seen in the screenshot it looks like the field "message" is not valid json.

---

<div class="post-metadata">

**Author:** ![ali\_ghorbani](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ali_ghorbani/32/84372_2.png) [@ali\_ghorbani](https://discuss.elastic.co/u/ali_ghorbani)\
**Post date:** [February 23, 2021, 11:36am UTC](https://discuss.elastic.co/t/logstash-parsing-json/265163/5 "2021-02-23T11:36:43Z")

</div>

this is the all contents of 1.log

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/d/f/df824ae62cffff22170f5be859a351787d6100c8.png)

---

<div class="post-metadata">

**Author:** ![grumo35](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grumo35/32/59451_2.png) [@grumo35](https://discuss.elastic.co/u/grumo35)\
**Post date:** [February 23, 2021, 11:45am UTC](https://discuss.elastic.co/t/logstash-parsing-json/265163/6 "2021-02-23T11:45:41Z")

</div>

Oh so here you have your error the values shown above does not represent a valid JSON.

Do you built it yourself ?

---

<div class="post-metadata">

**Author:** ![ali\_ghorbani](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ali_ghorbani/32/84372_2.png) [@ali\_ghorbani](https://discuss.elastic.co/u/ali_ghorbani)\
**Post date:** [February 23, 2021, 12:00pm UTC](https://discuss.elastic.co/t/logstash-parsing-json/265163/7 "2021-02-23T12:00:40Z")

</div>

yes i wrote a python script to build this file. but as i said before its behave randomly; sometime works and sometime not.

---

<div class="post-metadata">

**Author:** ![ali\_ghorbani](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ali_ghorbani/32/84372_2.png) [@ali\_ghorbani](https://discuss.elastic.co/u/ali_ghorbani)\
**Post date:** [February 24, 2021, 8:55am UTC](https://discuss.elastic.co/t/logstash-parsing-json/265163/8 "2021-02-24T08:55:21Z")

</div>

you right using a python script to produce the jsons solved the problem thank you.

---

<div class="post-metadata">

**Author:** ![grumo35](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grumo35/32/59451_2.png) [@grumo35](https://discuss.elastic.co/u/grumo35)\
**Post date:** [March 1, 2021, 10:23am UTC](https://discuss.elastic.co/t/logstash-parsing-json/265163/9 "2021-03-01T10:23:51Z")

</div>

Glad to help

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 29, 2021, 10:24am UTC](https://discuss.elastic.co/t/logstash-parsing-json/265163/10 "2021-03-29T10:24:31Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.