# S3 No files found in bucket

**URL:** <https://discuss.elastic.co/t/s3-no-files-found-in-bucket/282332>\
**Category:** Logstash\
**Created:** [August 24, 2021, 10:19am UTC](https://discuss.elastic.co/t/s3-no-files-found-in-bucket/282332 "2021-08-24T10:19:50Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![whassanwj](https://avatars.discourse-cdn.com/v4/letter/w/b782af/32.png) [@whassanwj](https://discuss.elastic.co/u/whassanwj)\
**Post date:** [August 24, 2021, 10:19am UTC](https://discuss.elastic.co/t/s3-no-files-found-in-bucket/282332/1 "2021-08-24T10:19:50Z")

</div>

Hi,  
I'm struggling with logstash conf to read from S3 bucket which has the latest date, example structure in S3 bucket as follows:

- apachelogs/web/2021-08-22/
- apachelogs/web/2021-08-23/
- apachelogs/web/2021-08-24/

Hence, my logstash input as follow

```auto
input {
    s3 {
        bucket => "apachelogs"
        prefix => "web/%{+YYYY-MM-dd}/"
        region => "ap-southeast-1"
        access_key_id => "xxxx"
        secret_access_key => "xxxx"
        sincedb_path => "/tmp/s3.sincedb"
        backup_to_dir => "/tmp/logstashed/"
        type => "apache"
    }
}

```

However, I kept getting  
`[INFO] 2021-08-24 18:04:06.804 [[main]<s3] s3 - No files found in bucket {:prefix=>"apachelogs/web/%{+YYYY-MM-dd}/"}` and interestingly with hardcoded date, it worked -\> `prefix=>"apachelogs/web/2021-08-24/"}`

Appreciate some help here. Thanks.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 24, 2021, 2:00pm UTC](https://discuss.elastic.co/t/s3-no-files-found-in-bucket/282332/2 "2021-08-24T14:00:00Z")

</div>

> [@whassanwj](#):
>
> `prefix => "web/%{+YYYY-MM-dd}/"`

A sprintf reference to a date uses the value of [@timestamp] from an event. This cannot be used in an input plugin since there are no events when the input is being configured. It will, as you have found, use the literal value.

---

<div class="post-metadata">

**Author:** ![whassanwj](https://avatars.discourse-cdn.com/v4/letter/w/b782af/32.png) [@whassanwj](https://discuss.elastic.co/u/whassanwj)\
**Post date:** [August 24, 2021, 3:26pm UTC](https://discuss.elastic.co/t/s3-no-files-found-in-bucket/282332/3 "2021-08-24T15:26:42Z")

</div>

Hi Badger, thanks for the response, so with this, is there any workaround i can do?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 24, 2021, 3:28pm UTC](https://discuss.elastic.co/t/s3-no-files-found-in-bucket/282332/4 "2021-08-24T15:28:39Z")

</div>

> [@whassanwj](#):
>
> is there any workaround i can do?

Start logstash once a day and pass the date using an [environment variable](https://www.elastic.co/guide/en/logstash/current/environment-variables.html).

---

<div class="post-metadata">

**Author:** ![whassanwj](https://avatars.discourse-cdn.com/v4/letter/w/b782af/32.png) [@whassanwj](https://discuss.elastic.co/u/whassanwj)\
**Post date:** [August 24, 2021, 3:33pm UTC](https://discuss.elastic.co/t/s3-no-files-found-in-bucket/282332/5 "2021-08-24T15:33:01Z")

</div>

> [@Badger](#):
>
> Start logstash once a day and pass the date using an [environment variable](https://www.elastic.co/guide/en/logstash/current/environment-variables.html).

Perfect, thanks a bunch!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 21, 2021, 3:33pm UTC](https://discuss.elastic.co/t/s3-no-files-found-in-bucket/282332/6 "2021-09-21T15:33:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.