S3 Output - Minio Connection Error

Elastic Stack Logstash
1

Hi! We're trying to output to Minio via the S3 output plugin. We specify the name of the server via endpoint: and we're getting this error:

[ERROR][logstash.outputs.s3 ] Uploading failed, retrying. {:exception=>Seahorse::Client::NetworkingError, :message=>"Failed to open TCP connection to click-data.minio.ourdomain.com:80 (initialize: name or service not known)"

click-data is the name of the bucket. minio.ourdomain.com exists and I can access it from within the container via port 80.

In fact, I can put the minio client in the Logstash container and access the server and bucket.

It looks like it's combining the bucket and the endpoint to create the server name. Here's the config:

	s3 {
		secret_access_key => "reallylongstring"
		access_key_id => "anotherreallylongstring"
		endpoint => "http://minio.ourdomain.com"
		bucket => "click-data"
		prefix => "%{+YYYY-MM}"
        ...otherstuff...
        }

Any ideas?

2

Thinking that the plugin is literally tryingto connect to http://bucket.endpoint:80 - I put that in the hosts file on the box (click-data.minio.ourdomain.com)

Now we get this error message:

[ERROR][logstash.outputs.s3      ] Uploading failed, retrying. {:exception=>Aws::S3::Errors::NoSuchBucket, :message=>"The specified bucket does not exist"

Which sort of confirms that. tcpdump show's the same, it's using the bucket name as part of the server to connect to via the endpoint.

3

I've confirmed that if we use a date prefix, that prefix becomes the bucket and uploads succeed. Not what we want at all and I don't believe this is how that plugin should work.

I've filed an issue on the github project page.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.