# S3 v4 signatures for logstash input plugin

**URL:** <https://discuss.elastic.co/t/s3-v4-signatures-for-logstash-input-plugin/46390>\
**Category:** Logstash\
**Created:** [April 5, 2016, 12:40pm UTC](https://discuss.elastic.co/t/s3-v4-signatures-for-logstash-input-plugin/46390 "2016-04-05T12:40:01Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Abey\_Thomas](https://avatars.discourse-cdn.com/v4/letter/a/73ab20/32.png) [@Abey\_Thomas](https://discuss.elastic.co/u/Abey_Thomas)\
**Post date:** [April 5, 2016, 12:40pm UTC](https://discuss.elastic.co/t/s3-v4-signatures-for-logstash-input-plugin/46390/1 "2016-04-05T12:40:01Z")

</div>

Hi ,

I was trying to collect all my AWS config logs from S3 via logstash and could see it list the filenames fine but once listing is complete it starts to download the S3 file and that fails with an error "Error: Requests specifying Server Side Encryption with AWS KMS managed keys require AWS Signature Version 4.". Once it gets this error it will restart the plugin and start listing the files again and then error.

Here is the complete log (replaced file paths)  
S3 input: Download remote file {:remote\_key=\>"logs/111/file\_brqvFXbF63.json.gz", :local\_filename=\>"/tmp/logstash/file\_brqvFXbF63.json.gz", :level=\>:debug, :file=\>"logstash/inputs/s3.rb", :line=\>"344", :method=\>"download\_remote\_file"}  
A plugin had an unrecoverable error. Will restart this plugin.  
Plugin: \<LogStash::Inputs::S3 bucket=\>"s3bucket", codec=\>\<LogStash::Codecs::JSON charset=\>"UTF-8"\>, interval=\>600, prefix=\>"logs/111/", region=\>"eu-west-1", use\_ssl=\>true, delete=\>false, temporary\_directory=\>"/tmp/logstash"\>  
Error: Requests specifying Server Side Encryption with AWS KMS managed keys require AWS Signature Version 4.  
Exception: AWS::S3::Errors::InvalidArgument  
Stack: /opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.66.0/lib/aws/core/client.rb:375:in `return_or_raise' /opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.66.0/lib/aws/core/client.rb:476:in`client\_request'  
(eval):3:in `get_object' /opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.66.0/lib/aws/s3/s3_object.rb:1371:in`get\_object'  
/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.66.0/lib/aws/s3/s3\_object.rb:1090:in `read' /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-s3-2.0.6/lib/logstash/inputs/s3.rb:346:in`download\_remote\_file'  
org/jruby/RubyIO.java:1183:in `open' /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-s3-2.0.6/lib/logstash/inputs/s3.rb:345:in`download\_remote\_file'  
/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-s3-2.0.6/lib/logstash/inputs/s3.rb:321:in `process_log' /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-s3-2.0.6/lib/logstash/inputs/s3.rb:151:in`process\_files'  
org/jruby/RubyArray.java:1613:in `each' /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-s3-2.0.6/lib/logstash/inputs/s3.rb:146:in`process\_files'  
/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-s3-2.0.6/lib/logstash/inputs/s3.rb:102:in `run' org/jruby/RubyProc.java:281:in`call'  
/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/interval.rb:20:in `interval' /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-s3-2.0.6/lib/logstash/inputs/s3.rb:101:in`run'  
/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.3-java/lib/logstash/pipeline.rb:331:in `inputworker' /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.3-java/lib/logstash/pipeline.rb:325:in`start\_input' {:level=\>:error, :file=\>"logstash/pipeline.rb", :line=\>"342", :method=\>"inputworker"}  
S3 input: Found key {:key=\>"logs/111/fil2\_akbtQDyyS6nHW.json.gz", :level=\>:debug, :file=\>"logstash/inputs/s3.rb", :line=\>"111", :method=\>"list\_new\_files"}

I can see in logstash-output-s3 plugin "Support S3 v4 signatures" was added some time this year. Does the input plugin has support for AWS S3 v4 signatures?

Regards,  
Abey

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 5, 2016, 11:08pm UTC](https://discuss.elastic.co/t/s3-v4-signatures-for-logstash-input-plugin/46390/2 "2016-04-05T23:08:04Z")

</div>

It doesn't look like it, it'd be good if you could raise a FR against the plugin on github 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:03am UTC](https://discuss.elastic.co/t/s3-v4-signatures-for-logstash-input-plugin/46390/3 "2017-07-06T05:03:40Z")

</div>