# S3 input, execution expired \[SOLVED\]

**URL:** <https://discuss.elastic.co/t/s3-input-execution-expired-solved/32996>\
**Category:** Logstash\
**Created:** [October 26, 2015, 3:01pm UTC](https://discuss.elastic.co/t/s3-input-execution-expired-solved/32996 "2015-10-26T15:01:55Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![calle\_eman](https://avatars.discourse-cdn.com/v4/letter/c/34f0e0/32.png) [@calle\_eman](https://discuss.elastic.co/u/calle_eman)\
**Post date:** [October 26, 2015, 3:01pm UTC](https://discuss.elastic.co/t/s3-input-execution-expired-solved/32996/1 "2015-10-26T15:01:56Z")

</div>

I have one logstash that's shipping files to s3, and one logstash that gets them from s3 and inserts them in elasticsearch. But the s3 input isn't working.

Config:

```
s3 {
  access_key_id => "XXXX"
  secret_access_key => "XXXX"
  region => "eu-west-1"
  bucket => "MyBucket"
  delete => true
}

```

When running logstash -f s3.conf --debug:

```
{:timestamp=>"2015-10-26T15:44:17.762000+0100", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::S3 access_key_id=>\"XXX\", secret_access_key=>\"XXXX\", region=>\"eu-west-1\", bucket=>\"MyBucket\", delete=>true, debug=>true, codec=><LogStash::Codecs::P
lain charset=>\"UTF-8\">, use_ssl=>true, interval=>60, temporary_directory=>\"/tmp/logstash\">\n Error: execution expired\n Exception: Timeout::Error\n Stack: org/jruby/ext/socket/RubyTC
PSocket.java:111:in `initialize'\norg/jruby/RubyIO.java:1179:in `open'\n/opt/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:763:in `connect'\norg/jruby/ext/timeout/Timeout.java:126:in `time
out'\n/opt/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:763:in `connect'\n/opt/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:756:in `do_start'\n/opt/logstash/vendor/jruby/lib/ruby/1.9/ne
t/http.rb:751:in `start'\n/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/core/http/connection_pool.rb:327:in `start_session'\n/opt/logstash/vendor/bundle/jruby/1.9/gem
s/aws-sdk-v1-1.64.0/lib/aws/core/http/connection_pool.rb:127:in `session_for'\n/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/core/http/net_http_handler.rb:56:in `hand
le'\n/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/core/client.rb:253:in `make_sync_request'\n/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/cor
e/client.rb:289:in `retry_server_errors'\n/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/s3/region_detection.rb:11:in `retry_server_errors'\n/opt/logstash/vendor/bundl
e/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/core/client.rb:249:in `make_sync_request'\n/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/core/client.rb:509:in `client_requ
est'\n/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/core/client.rb:391:in `log_client_request'\n/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/c
ore/client.rb:477:in `client_request'\n/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/core/client.rb:373:in `return_or_raise'\n/opt/logstash/vendor/bundle/jruby/1.9/ge
ms/aws-sdk-v1-1.64.0/lib/aws/core/client.rb:476:in `client_request'\n(eval):3:in `list_objects'\n/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/s3/object_collection.rb:298:in `list_request'\n/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/s3/paginated_collection.rb:29:in `_each_item'\n/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/core/collection/with_limit_and_next_token.rb:54:in `_each_batch'\n/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/core/collection.rb:80:in `each_batch'\n/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/core/collection.rb:47:in `each'\n/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/s3/object_collection.rb:282:in `each'\n/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-s3-1.0.0/lib/logstash/inputs/s3.rb:109:in `list_new_files'\n/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-s3-1.0.0/lib/logstash/inputs/s3.rb:143:in `process_files'\n/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-s3-1.0.0/lib/logstash/inputs/s3.rb:101:in `run'\norg/jruby/RubyProc.java:271:in `call'\n/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.21/lib/stud/interval.rb:17:in `interval'\n/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-s3-1.0.0/lib/logstash/inputs/s3.rb:100:in `run'\n/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/pipeline.rb:177:in `inputworker'\n/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/pipeline.rb:171:in `start_input'", :level=>:error, :file=>"logstash/pipeline.rb", :line=>"182", :method=>"inputworker"}

```

The credentials are working fine for the s3 output, and the policy is:

```
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": [
                "arn:aws:s3:::MyBucket/*",
                "arn:aws:s3:::MyBucket"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets"
            ],
            "Resource": [
                "arn:aws:s3:::*"
            ]
        }
    ]
}

```

Any ideas?

---

<div class="post-metadata">

**Author:** ![calle\_eman](https://avatars.discourse-cdn.com/v4/letter/c/34f0e0/32.png) [@calle\_eman](https://discuss.elastic.co/u/calle_eman)\
**Post date:** [October 26, 2015, 3:02pm UTC](https://discuss.elastic.co/t/s3-input-execution-expired-solved/32996/2 "2015-10-26T15:02:47Z")

</div>

The rest of the debug output:

```
org/jruby/ext/socket/RubyTCPSocket.java:111:in `initialize'
org/jruby/RubyIO.java:1179:in `open'
/opt/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:763:in `connect'
org/jruby/ext/timeout/Timeout.java:126:in `timeout'
/opt/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:763:in `connect'
/opt/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:756:in `do_start'
/opt/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:751:in `start'
/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/core/http/connection_pool.rb:327:in `start_session'
/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/core/http/connection_pool.rb:127:in `session_for'
/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/core/http/net_http_handler.rb:56:in `handle'
/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/core/client.rb:253:in `make_sync_request'
/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/core/client.rb:289:in `retry_server_errors'
/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/s3/region_detection.rb:11:in `retry_server_errors'
/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/core/client.rb:249:in `make_sync_request'
/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/core/client.rb:509:in `client_request'
/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/core/client.rb:391:in `log_client_request'
/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/core/client.rb:477:in `client_request'
/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/core/client.rb:373:in `return_or_raise'
/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/core/client.rb:476:in `client_request'
(eval):3:in `list_objects'
/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/s3/object_collection.rb:298:in `list_request'
/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/s3/paginated_collection.rb:29:in `_each_item'
/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/core/collection/with_limit_and_next_token.rb:54:in `_each_batch'
/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/core/collection.rb:80:in `each_batch'
/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/core/collection.rb:47:in `each'
/opt/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-v1-1.64.0/lib/aws/s3/object_collection.rb:282:in `each'
/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-s3-1.0.0/lib/logstash/inputs/s3.rb:109:in `list_new_files'
/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-s3-1.0.0/lib/logstash/inputs/s3.rb:143:in `process_files'
/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-s3-1.0.0/lib/logstash/inputs/s3.rb:101:in `run'
org/jruby/RubyProc.java:271:in `call'
/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.21/lib/stud/interval.rb:17:in `interval'
/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-s3-1.0.0/lib/logstash/inputs/s3.rb:100:in `run'
/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/pipeline.rb:177:in `inputworker'
/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/pipeline.rb:171:in `start_input'
{:timestamp=>"2015-10-26T15:44:17.822000+0100", :message=>"Plugin is finished", :plugin=><LogStash::Inputs::S3 access_key_id=>"XXXX", secret_access_key=>"XXXX", region=>"eu-west-1", bucket=>"MyBucket", delete=>true, debug=>true, codec=><LogStash::Codecs::Plain charset=>"UTF-8">, use_ssl=>true, interval=>60, temporary_directory=>"/tmp/logstash">, :level=>:info, :file=>"logstash/plugin.rb", :line=>"61", :method=>"finished"}
```

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [October 27, 2015, 1:34am UTC](https://discuss.elastic.co/t/s3-input-execution-expired-solved/32996/3 "2015-10-27T01:34:42Z")

</div>

What version of LS are you using?

---

<div class="post-metadata">

**Author:** ![calle\_eman](https://avatars.discourse-cdn.com/v4/letter/c/34f0e0/32.png) [@calle\_eman](https://discuss.elastic.co/u/calle_eman)\
**Post date:** [October 27, 2015, 7:39am UTC](https://discuss.elastic.co/t/s3-input-execution-expired-solved/32996/4 "2015-10-27T07:39:07Z")

</div>

Logstash 1.5.4

---

<div class="post-metadata">

**Author:** ![calle\_eman](https://avatars.discourse-cdn.com/v4/letter/c/34f0e0/32.png) [@calle\_eman](https://discuss.elastic.co/u/calle_eman)\
**Post date:** [October 28, 2015, 8:10am UTC](https://discuss.elastic.co/t/s3-input-execution-expired-solved/32996/5 "2015-10-28T08:10:58Z")

</div>

It seems [someone else has gotten this error before](http://techblog.mdsol.com/2014/01/27/parsing-amazon-cloudtrail-json-logs-with-a-customized-logstash-build.html#connecting-to-s3). But using s3fog instead of s3 seems like a roundabout way of doing it. There are no http-proxys involved here anywhere.

---

<div class="post-metadata">

**Author:** ![calle\_eman](https://avatars.discourse-cdn.com/v4/letter/c/34f0e0/32.png) [@calle\_eman](https://discuss.elastic.co/u/calle_eman)\
**Post date:** [October 28, 2015, 10:41am UTC](https://discuss.elastic.co/t/s3-input-execution-expired-solved/32996/6 "2015-10-28T10:41:41Z")

</div>

Turns out it was a connectivity issue. Sorry for the confusion 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:25am UTC](https://discuss.elastic.co/t/s3-input-execution-expired-solved/32996/7 "2017-07-06T05:25:05Z")

</div>