# Logstash s3 output

**URL:** https://discuss.elastic.co/t/logstash-s3-output/311118
**Category:** Logstash
**Created:** [August 1, 2022, 1:09pm UTC](https://discuss.elastic.co/t/logstash-s3-output/311118 "2022-08-01T13:09:41Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![Giridharan\_C](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/giridharan_c/32/102396_2.png) [@Giridharan\_C](https://discuss.elastic.co/u/Giridharan_C)
#### Post date: [August 1, 2022, 1:09pm UTC](https://discuss.elastic.co/t/logstash-s3-output/311118/1 "2022-08-01T13:09:41Z")

</div>

My logstash s3 output is not working as defined in the pipeline config. The exact output config is working for other pipelines. The dynamic directory structure(/syslog/%{+YYYY}/%{+MM}/%{+dd}/%{+HH}) defined in the config is not created.  
The expected directory structure "s3://my-s3-bucket/syslog/2022/08/12/filename.txt.gz".  
What I see in the s3 is "s3://my-s3-bucket/syslog////filename.txt.gz"

output:

```auto
output {
  s3 {
    id => "syslog_id"
    bucket => "my-s3-bucket"
    prefix => "/syslog/%{+YYYY}/%{+MM}/%{+dd}/%{+HH}"
    temporary_directory => "/app/syslog_temp"
    encoding => "gzip"
    codec => json_lines
  }
}

```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 1, 2022, 1:15pm UTC](https://discuss.elastic.co/t/logstash-s3-output/311118/2 "2022-08-01T13:15:25Z")

</div>

Have you removed the [@timestamp] field from your events?

---

<div class="post-metadata">

### Author: ![Giridharan\_C](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/giridharan_c/32/102396_2.png) [@Giridharan\_C](https://discuss.elastic.co/u/Giridharan_C)
#### Post date: [August 1, 2022, 1:20pm UTC](https://discuss.elastic.co/t/logstash-s3-output/311118/3 "2022-08-01T13:20:08Z")

</div>

yes... I see it is repeating field from the syslog header.

```auto
mutate {
                remove_field => ["@timestamp"]
        }
```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 1, 2022, 2:11pm UTC](https://discuss.elastic.co/t/logstash-s3-output/311118/4 "2022-08-01T14:11:07Z")

</div>

> [@Giridharan\_C](#):
>
> `%{+YYYY}/%{+MM}/%{+dd}/%{+HH}`

Those are expanded using the [@timestamp] field. If the field does not exist then they will be empty strings.

---

<div class="post-metadata">

### Author: ![Giridharan\_C](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/giridharan_c/32/102396_2.png) [@Giridharan\_C](https://discuss.elastic.co/u/Giridharan_C)
#### Post date: [August 2, 2022, 8:59am UTC](https://discuss.elastic.co/t/logstash-s3-output/311118/5 "2022-08-02T08:59:12Z")

</div>

> [@Badger](#):
>
> Those are expanded using the [@timestamp] field. If the field does not exist then they will be empty strings.

Thank you @Badger. Now the config is working as defined.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 30, 2022, 8:59am UTC](https://discuss.elastic.co/t/logstash-s3-output/311118/6 "2022-08-30T08:59:15Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
