# Logstash s3 plugin to push json files to elasticsearch

**URL:** <https://discuss.elastic.co/t/logstash-s3-plugin-to-push-json-files-to-elasticsearch/306325>\
**Category:** Logstash\
**Created:** [June 3, 2022, 10:43am UTC](https://discuss.elastic.co/t/logstash-s3-plugin-to-push-json-files-to-elasticsearch/306325 "2022-06-03T10:43:30Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![JinnaBalu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jinnabalu/32/27899_2.png) [@JinnaBalu](https://discuss.elastic.co/u/JinnaBalu)\
**Post date:** [June 3, 2022, 10:43am UTC](https://discuss.elastic.co/t/logstash-s3-plugin-to-push-json-files-to-elasticsearch/306325/1 "2022-06-03T10:43:30Z")

</div>

**Usecase**

Push s3 JSON objects (s3 containers json files), each object will be one document)

I have written a pipeline

```auto
input {
    s3 {
        access_key_id => "MY_KEY"
        secret_access_key => "MY_SECRET"
        bucket => "sthreetoes"
        region => "ap-south-1"
        codec => "json"
    }
}
output {
    elasticsearch {
        hosts => "http://elasticsearch:9200"
        index => "candidate-data"
    }
}

```

When I run the pipeline, losgstash is shipping to index considering each key-value paid as document.

json object in s3

```json
{
    "id": 246,
    "first_name": "Lell",
    "last_name": "Bsel",
    "email": "lbros6t@sakura.jp",
    "gender": "Female",
    "ip_address": "12.12.12.8"
}

```

When we run the logstash pipeline index is created with 6 documents considering each key-value pair in JSON as a message.

Any solution to save the JSON object as a single document

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 3, 2022, 3:46pm UTC](https://discuss.elastic.co/t/logstash-s3-plugin-to-push-json-files-to-elasticsearch/306325/2 "2022-06-03T15:46:21Z")

</div>

It sounds like you need a multiline codec on the input.

---

<div class="post-metadata">

**Author:** ![JinnaBalu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jinnabalu/32/27899_2.png) [@JinnaBalu](https://discuss.elastic.co/u/JinnaBalu)\
**Post date:** [June 6, 2022, 5:25am UTC](https://discuss.elastic.co/t/logstash-s3-plugin-to-push-json-files-to-elasticsearch/306325/3 "2022-06-06T05:25:41Z")

</div>

Yes, we are going with one line will not help pickup the whole json.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 4, 2022, 5:26am UTC](https://discuss.elastic.co/t/logstash-s3-plugin-to-push-json-files-to-elasticsearch/306325/4 "2022-07-04T05:26:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.