# Error in parsing Pretty Json Data

**URL:** <https://discuss.elastic.co/t/error-in-parsing-pretty-json-data/293103>\
**Category:** Logstash\
**Created:** [December 29, 2021, 10:46am UTC](https://discuss.elastic.co/t/error-in-parsing-pretty-json-data/293103 "2021-12-29T10:46:48Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Aniket\_Pant](https://avatars.discourse-cdn.com/v4/letter/a/77aa72/32.png) [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\
**Post date:** [December 29, 2021, 10:46am UTC](https://discuss.elastic.co/t/error-in-parsing-pretty-json-data/293103/1 "2021-12-29T10:46:48Z")

</div>

Hi team

I have file in json format.

```auto
[
    {
        "year": 2013,
        "title": "Rush",
        "info": {
            "directors": ["Ron Howard"],
            "release_date": "2013-09-02T00:00:00Z",
            "rating": 8.3,
            "genres": [
                "Action",
                "Biography",
                "Drama",
                "Sport"
            ],
            "image_url": "http://ia.media-imdb.com/images/M/MV5BMTQyMDE0MTY0OV5BMl5BanBnXkFtZTcwMjI2OTI0OQ@@._V1_SX400_.jpg",
            "plot": "A re-creation of the merciless 1970s rivalry between Formula One rivals James Hunt and Niki Lauda.",
            "rank": 2,
            "running_time_secs": 7380,
            "actors": [
                "Daniel Bruhl",
                "Chris Hemsworth",
                "Olivia Wilde"
            ]
        }
    }
]

```

i want to parse this file and i am dealing with error in **logstash json filter**

```auto
input
{
file {
 path => "/usr/share/logstash/moviedata.json"
 start_position => "beginning"
 #sincedb_path => "/usr/share/logstash/dbteste"
}
}

filter {
json {
source => "message"
}
}

```

**logstash logs**

```auto
[2021-12-29T16:01:13,824][WARN][logstash.filters.json][movie][2e0349d64f8ab2889caf33284adb464eb2adf3d4b19e77ad5922b144d2a77e57] Error parsing json {:source=>"message", :raw=>" }", :exception=>#<LogStash::Json::ParserError: Unexpected close marker '}': expected ']' (for root starting at [Source: (byte[])" }"; line: 1, column: 0])
 at [Source: (byte[])" }"; line: 1, column: 6]>}
[2021-12-29T16:01:13,825][WARN][logstash.filters.json][movie][2e0349d64f8ab2889caf33284adb464eb2adf3d4b19e77ad5922b144d2a77e57] Error parsing json {:source=>"message", :raw=>" \"Action\",", :exception=>#<LogStash::Json::ParserError: Unexpected character (',' (code 44)): expected a value
 at [Source: (byte[])" "Action","; line: 1, column: 26]>}
[2021-12-29T16:01:13,830][WARN][logstash.filters.json][movie][2e0349d64f8ab2889caf33284adb464eb2adf3d4b19e77ad5922b144d2a77e57] Error parsing json {:source=>"message", :raw=>" \"Biography\",", :exception=>#<LogStash::Json::ParserError: Unexpected character (',' (code 44)): expected a value
 at [Source: (byte[])" "Biography","; line: 1, column: 29]>}
[2021-12-29T16:01:13,836][WARN][logstash.filters.json][movie][2e0349d64f8ab2889caf33284adb464eb2adf3d4b19e77ad5922b144d2a77e57] Error parsing json {:source=>"message", :raw=>" \"plot\": \"A re-creation of the merciless 1970s rivalry between Formula One rivals James Hunt and Niki Lauda.\",", :exception=>#<LogStash::Json::ParserError: Unexpected character (':' (code 58)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: (byte[])" "plot": "A re-creation of the merciless 1970s rivalry between Formula One rivals James Hunt and Niki Lauda.","; line: 1, column: 20]>}
[2021-12-29T16:01:13,837][WARN][logstash.filters.json][movie][2e0349d64f8ab2889caf33284adb464eb2adf3d4b19e77ad5922b144d2a77e57] Error parsing json {:source=>"message", :raw=>" \"Daniel Bruhl\",", :exception=>#<LogStash::Json::ParserError: Unexpected character (',' (code 44)): expected a value
 at [Source: (byte[])" "Daniel Bruhl","; line: 1, column: 32]>}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [December 29, 2021, 5:03pm UTC](https://discuss.elastic.co/t/error-in-parsing-pretty-json-data/293103/2 "2021-12-29T17:03:08Z")

</div>

You need to use a multiline codec to combine the pretty-printed JSON into a single event. There is an example of that [here](https://discuss.elastic.co/t/parsing-xml-log-file-with-logstash/167306/2).

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 29, 2021, 5:26pm UTC](https://discuss.elastic.co/t/error-in-parsing-pretty-json-data/293103/3 "2021-12-29T17:26:53Z")

</div>

Another possible approach is to simply turn your data into `ndjson` (new line delimited json) using `jq` found [here](https://stedolan.github.io/jq/) then you would not need to worry about the multiline etc.

It is a super powerful json tool...

Say your file looks like this ( I put 2 in but it will work with 1 to n)

```auto
$ cat json-pretty-sample.json 
[
  {
    "year": 2013,
    "title": "Rush",
    "info": {
      "directors": [
        "Ron Howard"
      ],
      "release_date": "2013-09-02T00:00:00Z",
      "rating": 8.3,
      "genres": [
        "Action",
        "Biography",
        "Drama",
        "Sport"
      ],
      "image_url": "http://ia.media-imdb.com/images/M/MV5BMTQyMDE0MTY0OV5BMl5BanBnXkFtZTcwMjI2OTI0OQ@@._V1_SX400_.jpg",
      "plot": "A re-creation of the merciless 1970s rivalry between Formula One rivals James Hunt and Niki Lauda.",
      "rank": 2,
      "running_time_secs": 7380,
      "actors": [
        "Daniel Bruhl",
        "Chris Hemsworth",
        "Olivia Wilde"
      ]
    }
  },
  {
    "year": 2015,
    "title": "Other Movie",
    "info": {
      "directors": [
        "Ron Howard"
      ],
      "release_date": "2013-09-02T00:00:00Z",
      "rating": 8.3,
      "genres": [
        "Action",
        "Biography",
        "Drama",
        "Sport"
      ],
      "image_url": "http://ia.media-imdb.com/images/M/MV5BMTQyMDE0MTY0OV5BMl5BanBnXkFtZTcwMjI2OTI0OQ@@._V1_SX400_.jpg",
      "plot": "A re-creation of the merciless 1970s rivalry between Formula One rivals James Hunt and Niki Lauda.",
      "rank": 2,
      "running_time_secs": 7380,
      "actors": [
        "Daniel Bruhl",
        "Chris Hemsworth",
        "Olivia Wilde"
      ]
    }
  }
]

```

I can simply run `jq` and tell it to write `ndjson`

This command says... `jq` output in compact form (ndjson) `-c` and write all the elements within the top array `.[]`

```auto
$ cat json-pretty-sample.json | jq -c .[] > sample.ndjson

```

and now the file will be ndjson which is what logstash can easily read without the multiline code

```auto
$ cat sample.ndjson 
{"year":2013,"title":"Rush","info":{"directors":["Ron Howard"],"release_date":"2013-09-02T00:00:00Z","rating":8.3,"genres":["Action","Biography","Drama","Sport"],"image_url":"http://ia.media-imdb.com/images/M/MV5BMTQyMDE0MTY0OV5BMl5BanBnXkFtZTcwMjI2OTI0OQ@@._V1_SX400_.jpg","plot":"A re-creation of the merciless 1970s rivalry between Formula One rivals James Hunt and Niki Lauda.","rank":2,"running_time_secs":7380,"actors":["Daniel Bruhl","Chris Hemsworth","Olivia Wilde"]}}
{"year":2015,"title":"Other Movie","info":{"directors":["Ron Howard"],"release_date":"2013-09-02T00:00:00Z","rating":8.3,"genres":["Action","Biography","Drama","Sport"],"image_url":"http://ia.media-imdb.com/images/M/MV5BMTQyMDE0MTY0OV5BMl5BanBnXkFtZTcwMjI2OTI0OQ@@._V1_SX400_.jpg","plot":"A re-creation of the merciless 1970s rivalry between Formula One rivals James Hunt and Niki Lauda.","rank":2,"running_time_secs":7380,"actors":["Daniel Bruhl","Chris Hemsworth","Olivia Wilde"]}}

```

This conf read the ndjson file and parses everything just fine.  
Note I used the `json` codec to read the file.. I am often confused between the two `json` and `json_lines` but for reading from a file with ndjson ... `codec => "json"` gets the job done... you can also use the `filter` with `json` but this is pretty direct.

```auto
input {
  file {
    path => "/Users/sbrown/workspace/sample-data/discuss/logstash/sample.ndjson"
    start_position => "beginning"
    codec => "json"
    sincedb_path => "/dev/null"
  }
}

output {
	stdout { codec => rubydebug }
}

```

Output

```auto
{
          "year" => 2015,
         "title" => "Other Movie",
      "@version" => "1",
          "info" => {
                   "genres" => [
            [0] "Action",
            [1] "Biography",
            [2] "Drama",
            [3] "Sport"
        ],
                     "rank" => 2,
        "running_time_secs" => 7380,
                   "actors" => [
            [0] "Daniel Bruhl",
            [1] "Chris Hemsworth",
            [2] "Olivia Wilde"
        ],
             "release_date" => "2013-09-02T00:00:00Z",
                "image_url" => "http://ia.media-imdb.com/images/M/MV5BMTQyMDE0MTY0OV5BMl5BanBnXkFtZTcwMjI2OTI0OQ@@._V1_SX400_.jpg",
                "directors" => [
            [0] "Ron Howard"
        ],
                   "rating" => 8.3,
                     "plot" => "A re-creation of the merciless 1970s rivalry between Formula One rivals James Hunt and Niki Lauda."
    },
    "@timestamp" => 2021-12-29T17:59:23.458Z,
          "host" => "hyperion",
          "path" => "/Users/sbrown/workspace/sample-data/discuss/logstash/sample.ndjson"
}
.......

```

---

<div class="post-metadata">

**Author:** ![Aniket\_Pant](https://avatars.discourse-cdn.com/v4/letter/a/77aa72/32.png) [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\
**Post date:** [December 30, 2021, 6:51am UTC](https://discuss.elastic.co/t/error-in-parsing-pretty-json-data/293103/4 "2021-12-30T06:51:23Z")

</div>

> [@stephenb](#):
>
> `sincedb_path => "/dev/null"`

Hey @stephenb it is working now and thanks for this new command 'jq'

---

<div class="post-metadata">

**Author:** ![Aniket\_Pant](https://avatars.discourse-cdn.com/v4/letter/a/77aa72/32.png) [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\
**Post date:** [December 30, 2021, 6:51am UTC](https://discuss.elastic.co/t/error-in-parsing-pretty-json-data/293103/5 "2021-12-30T06:51:43Z")

</div>

thanks @Badger

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 27, 2022, 6:51am UTC](https://discuss.elastic.co/t/error-in-parsing-pretty-json-data/293103/6 "2022-01-27T06:51:46Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.