# Prettified json is not parsed by logstash s3 input plugin

**URL:** <https://discuss.elastic.co/t/prettified-json-is-not-parsed-by-logstash-s3-input-plugin/273398>\
**Category:** Logstash\
**Created:** [May 19, 2021, 11:32am UTC](https://discuss.elastic.co/t/prettified-json-is-not-parsed-by-logstash-s3-input-plugin/273398 "2021-05-19T11:32:11Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Anubha](https://avatars.discourse-cdn.com/v4/letter/a/e19b73/32.png) [@Anubha](https://discuss.elastic.co/u/Anubha)\
**Post date:** [May 19, 2021, 11:32am UTC](https://discuss.elastic.co/t/prettified-json-is-not-parsed-by-logstash-s3-input-plugin/273398/1 "2021-05-19T11:32:11Z")

</div>

We're trying to parse multiline json file from an s3 bucket which results in "\_jsonparsefailure".  
It reads the file line by line. The codec we're using is `json_lines`.

Our logstash config looks like this:

```
input {
      s3 {
        bucket => "${S3_BUCKET_NAME}"
        region => "${AWS_REGION}"
        codec => json_lines
      }
    }
    filter {
      split {
        field => "fieldName"
      }
    }
output { # elasticsearch config
} 

```

The file has only one json object and the fieldName in this case is an array inside the json object.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 19, 2021, 4:10pm UTC](https://discuss.elastic.co/t/prettified-json-is-not-parsed-by-logstash-s3-input-plugin/273398/2 "2021-05-19T16:10:10Z")

</div>

The json\_lines codec expects each line to be a complete JSON object. If your JSON object is pretty-printed across multiple lines you will need to use a multiline codec.

---

<div class="post-metadata">

**Author:** ![Anubha](https://avatars.discourse-cdn.com/v4/letter/a/e19b73/32.png) [@Anubha](https://discuss.elastic.co/u/Anubha)\
**Post date:** [May 19, 2021, 5:14pm UTC](https://discuss.elastic.co/t/prettified-json-is-not-parsed-by-logstash-s3-input-plugin/273398/3 "2021-05-19T17:14:39Z")

</div>

@Badger Thanks for your reply. With large files, wouldn't multiline codec reach a limit? I remember it was breaking down the file since it was too large and then the json would then not make sense

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 19, 2021, 5:48pm UTC](https://discuss.elastic.co/t/prettified-json-is-not-parsed-by-logstash-s3-input-plugin/273398/4 "2021-05-19T17:48:44Z")

</div>

The multiline codec has [options](https://www.elastic.co/guide/en/logstash/current/plugins-codecs-multiline.html#plugins-codecs-multiline-max_bytes) to set the limit on the number of bytes and lines that can be combined. The defaults are 500 lines and 10 megabytes. You are free to increase them if you need to.

---

<div class="post-metadata">

**Author:** ![Anubha](https://avatars.discourse-cdn.com/v4/letter/a/e19b73/32.png) [@Anubha](https://discuss.elastic.co/u/Anubha)\
**Post date:** [May 19, 2021, 6:01pm UTC](https://discuss.elastic.co/t/prettified-json-is-not-parsed-by-logstash-s3-input-plugin/273398/5 "2021-05-19T18:01:03Z")

</div>

Great! Thanks a lot @Badger

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 16, 2021, 6:01pm UTC](https://discuss.elastic.co/t/prettified-json-is-not-parsed-by-logstash-s3-input-plugin/273398/6 "2021-06-16T18:01:23Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.