# Logstash S3 output requires bucket object permissions at the root of the bucket, regardless of prefix config

**URL:** <https://discuss.elastic.co/t/logstash-s3-output-requires-bucket-object-permissions-at-the-root-of-the-bucket-regardless-of-prefix-config/293951>\
**Category:** Logstash\
**Created:** [January 10, 2022, 10:30pm UTC](https://discuss.elastic.co/t/logstash-s3-output-requires-bucket-object-permissions-at-the-root-of-the-bucket-regardless-of-prefix-config/293951 "2022-01-10T22:30:54Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![mdebord](https://avatars.discourse-cdn.com/v4/letter/m/e9c0ed/32.png) [@mdebord](https://discuss.elastic.co/u/mdebord)\
**Post date:** [January 10, 2022, 10:30pm UTC](https://discuss.elastic.co/t/logstash-s3-output-requires-bucket-object-permissions-at-the-root-of-the-bucket-regardless-of-prefix-config/293951/1 "2022-01-10T22:30:54Z")

</div>

Logstash requires write access to the root of the bucket, regardless of the prefix defined in config.

There are a few issues to note with this behavior:

- Logstash should be isolating its work into the defined prefix path in config, but it's not.

- I'm forced to give logstash write access to the **root** of the bucket! (`"Resource": "arn:aws:s3:::my-log-bucket/*"`) regardless of the prefix setting.

- After starting logstash it creates test-write files in the root of the bucket, **not in the defined prefix path.** The logs themselves go in the prefix, these test files should too.

- Even though the "root" check can be disabled, ([S3 output plugin | Logstash Reference [7.16] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-s3.html#plugins-outputs-s3-validate_credentials_on_root_bucket)) however I don't want to disable permission checking overall.

Bucket contents:

```auto
logstash-programmatic-access-test-object-2022-01-10 21:44:46 +0000
logstash-programmatic-access-test-object-2022-01-10 21:58:05 +0000
logstash-programmatic-access-test-object-2022-01-10 21:59:34 +0000
logstash-programmatic-access-test-object-2022-01-10 22:01:27 +0000
logstash-programmatic-access-test-object-2022-01-10 22:04:12 +0000

logstash/...

```

Error thrown:

```auto
LogStash::ConfigurationError: Logstash must have the privileges to write to root bucket `my-log-bucket`, check your credentials or your permissions.>

```

Below is the config, that "should" work as-is, but since Logstash isn't respecting the prefix, the above error is shown and logstash doesn't start.

Output config looks like this:

```auto
output {
  s3 {
    region => "us-east-1"
    bucket => "my-log-bucket"
    prefix => "logstash/"
    encoding => "gzip"
    server_side_encryption => true
    server_side_encryption_algorithm => "aws:kms"
    size_file => 5242880
    time_file => 30
  }
}

```

An access policy such as this should suffice, but it doesn't. The entire root object needs to be granted, not just the prefix.

```auto
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::my-log-bucket"
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject"
            ],
            "Resource": "arn:aws:s3:::my-log-bucket/logstash/*"
        }
    ]
}

```

Granting it write access to `"arn:aws:s3:::my-log-bucket/*"` puts a band-aid on the problem but exposes more permissions than should be needed for any given service.

This setting [S3 output plugin | Logstash Reference [7.16] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-s3.html#plugins-outputs-s3-validate_credentials_on_root_bucket) appears to just disable the check completely which is not ideal.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 7, 2022, 10:31pm UTC](https://discuss.elastic.co/t/logstash-s3-output-requires-bucket-object-permissions-at-the-root-of-the-bucket-regardless-of-prefix-config/293951/2 "2022-02-07T22:31:15Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.