# Logstash S3 Input Plugin Error

**URL:** <https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245201>\
**Category:** Logstash\
**Created:** [August 17, 2020, 9:01am UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245201 "2020-08-17T09:01:14Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![elasticus3r](https://avatars.discourse-cdn.com/v4/letter/e/41988e/32.png) [@elasticus3r](https://discuss.elastic.co/u/elasticus3r)\
**Post date:** [August 17, 2020, 9:01am UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245201/1 "2020-08-17T09:01:14Z")

</div>

Hello All,

Hoping someone could help please.

I have got a simple ELk stack running and using s3 input plugin to ingest from an endpoint with the following config:

```auto
input {
  s3 {
    bucket => "mybucket"
    endpoint => "https://<object_storage_namespace>.compat.objectstorage.<region>.oraclecloud.com"
    region => "uk-london-1"
    access_key_id => " ************"
    secret_access_key => " *************"
    delete => false
    interval => 300 # seconds
    add_field => { "service" => "oci" }
    codec => "json"
  }
}

```

```auto
output {
  if [service] == "oci" {
    elasticsearch {
      hosts => ["http://localhost:9200"]
      index => "logstash-oci-%{+YYYY.MM}"
    }
  }
}

```

I am not getting any errors in the logstash logs but I am seeing this in Kibana discovery json document.

```auto
{
  "_index": "logstash-oci-2020.08",
  "_type": "_doc",
  "_id": "db-j-HMB8Q9kUzB1NjQQ",
  "_version": 1,
  "_score": null,
  "_source": {
    "@timestamp": "2020-08-16T18:58:32.170Z",
    "message": "}\n",
    "service": "oci",
    "@version": "1",
    "tags": [
      "_jsonparsefailure"
    ]
  },
  "fields": {
    "@timestamp": [
      "2020-08-16T18:58:32.170Z"
    ]
  },
  "sort": [
    1597604312170
  ]
}

```

I am using latest ELk Version and s3 input plugin versions.  
At the endpoint storage, I have uploaded a file with json logs called samplelog.log but for whatever reason the s3 plugin is not reading the log file correctly. As you can see in the message field, only see **"}\n"** and getting **"\_jsonparsefailure"** in tags.

Any ideas? Thanks.

---

<div class="post-metadata">

**Author:** ![elasticus3r](https://avatars.discourse-cdn.com/v4/letter/e/41988e/32.png) [@elasticus3r](https://discuss.elastic.co/u/elasticus3r)\
**Post date:** [August 17, 2020, 9:47am UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245201/2 "2020-08-17T09:47:53Z")

</div>

Okay with further testing, I have made some changes as described below:

1. Uploaded to the endpoint storage with another file called **example2.json** with the contents:

```auto
{
    "fruit": "Apple",
    "size": "Large",
    "color": "Red"
}

```

1. Restarted logstash and now seeing the errors in logstash logs:

[ERROR][logstash.codecs.json][main][cdeec7eab53d0d27aac4410853599e0063c9bb4baa4f5d84ff5faf279c170855] JSON parse error, original data now in message field {:error=\>#\<LogStash::Json::ParserError: Unexpected close marker '}': expected ']' (for root starting at [Source: (String)"}"; line: 1, column: 0])  
at [Source: (String)"}"; line: 1, column: 2]\>, :data=\>"}"}

Though format of file is in json, the logstash plugin still complains it is not in json. ☹

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 17, 2020, 2:02pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245201/3 "2020-08-17T14:02:54Z")

</div>

It looks like your files are pretty-printed JSON. An s3 input, like a file input, consumes the file one line at a time. So that example will be five events '{',  
'"fruit": "Apple",', '"size": "Large",', '"color": "Red"', and '}', none of which are valid JSON. You may be able to combine the parts of a single JSON object using a multiline codec.

---

<div class="post-metadata">

**Author:** ![elasticus3r](https://avatars.discourse-cdn.com/v4/letter/e/41988e/32.png) [@elasticus3r](https://discuss.elastic.co/u/elasticus3r)\
**Post date:** [August 17, 2020, 2:32pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245201/4 "2020-08-17T14:32:33Z")

</div>

Ahh thanks Badger.

So in this case if I put the json content in one line, the s3 plugin should read it accordingly and display in correctly in Kibana?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 17, 2020, 2:35pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245201/5 "2020-08-17T14:35:06Z")

</div>

I would expect so, yes.

---

<div class="post-metadata">

**Author:** ![elasticus3r](https://avatars.discourse-cdn.com/v4/letter/e/41988e/32.png) [@elasticus3r](https://discuss.elastic.co/u/elasticus3r)\
**Post date:** [August 17, 2020, 2:55pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245201/6 "2020-08-17T14:55:13Z")

</div>

Perfect that worked. Thank you!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 14, 2020, 2:55pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-error/245201/7 "2020-09-14T14:55:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.