# Logstash w/ s3 output plugin - slow/delay

**URL:** <https://discuss.elastic.co/t/logstash-w-s3-output-plugin-slow-delay/333836>\
**Category:** Logstash\
**Tags:** docker\
**Created:** [May 19, 2023, 4:05am UTC](https://discuss.elastic.co/t/logstash-w-s3-output-plugin-slow-delay/333836 "2023-05-19T04:05:31Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![alexus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexus/32/12696_2.png) [@alexus](https://discuss.elastic.co/u/alexus)\
**Post date:** [May 19, 2023, 4:05am UTC](https://discuss.elastic.co/t/logstash-w-s3-output-plugin-slow-delay/333836/1 "2023-05-19T04:05:31Z")

</div>

Hello,

I'm using Logstash 7.17.10 with [S3 output plugin](https://www.elastic.co/guide/en/logstash/7.17/plugins-outputs-s3.html), and getting poor performance (possibly not related to performance)

my pipeline:

```auto
input {
    elasticsearch {
        docinfo => true
        docinfo_fields => ["_id"]
        hosts => ["${ELASTICSEARCH_HOSTS}"]
        index => 'index*'
        password => "${ELASTICSEARCH_PASSWORD}"
        query => ' ... '
        schedule => '0 * * * *'
        user => "${ELASTICSEARCH_USER}"
    }
}

filter {
    ruby { code => 'event.set("[@metadata][suffix]", Time.now.strftime("%Y-%m-%dT%H"));' }
}

output {
    s3 {
        access_key_id => "${MINIO_ACCESS_KEY}"
        additional_settings => {
            force_path_style => true
        }
        bucket => "${MINIO_BUCKET}"
        codec => json_lines {
            target => ""
        }
        endpoint => "${MINIO_ENDPOINT}"
        prefix => "a-b-c-d/e-%{[@metadata][_index]}-%{[@metadata][suffix]}.json"
        secret_access_key => "${MINIO_SECRET_KEY}"
        validate_credentials_on_root_bucket => false
    }
}

```

it can easily 30+ minutes before i see objects on my s3 storage.

also both `elasticsearch` and `logstash` are residing on same network and so as minio (storage) as well and if i configure logstash to output file locally, it doesn't takes long at all to complete, and i'm able to push file manually to s3 quick as well (so it's not network).

please advise.  
thank you in advance!

---

<div class="post-metadata">

**Author:** ![alexus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexus/32/12696_2.png) [@alexus](https://discuss.elastic.co/u/alexus)\
**Post date:** [May 19, 2023, 12:15pm UTC](https://discuss.elastic.co/t/logstash-w-s3-output-plugin-slow-delay/333836/2 "2023-05-19T12:15:03Z")

</div>

does it have anything to do with [time\_file](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-s3.html#plugins-outputs-s3-time_file) ?

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [May 19, 2023, 12:32pm UTC](https://discuss.elastic.co/t/logstash-w-s3-output-plugin-slow-delay/333836/3 "2023-05-19T12:32:12Z")

</div>

> [@alexus](#):
>
> does it have anything to do with [time\_file](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-s3.html#plugins-outputs-s3-time_file) ?

Could be, the default `rotation_strategy` is `size_and_time`, so a file will only be pushed to s3 after it reaches the specified time (15 minutes per default) or the specified size ( ~ 5 MB per default).

But also keep in mind the the this output plugin only officially supports AWS S3, any S3 compatible it is not supported, it may work, but it also may have issues.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 16, 2023, 12:32pm UTC](https://discuss.elastic.co/t/logstash-w-s3-output-plugin-slow-delay/333836/4 "2023-06-16T12:32:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.