# Logstash S3 inout plugin crashes when reading Glacier object

**URL:** https://discuss.elastic.co/t/logstash-s3-inout-plugin-crashes-when-reading-glacier-object/141822
**Category:** Logstash
**Created:** [July 26, 2018, 5:17pm UTC](https://discuss.elastic.co/t/logstash-s3-inout-plugin-crashes-when-reading-glacier-object/141822 "2018-07-26T17:17:00Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![Hanciv](https://avatars.discourse-cdn.com/v4/letter/h/ea666f/32.png) [@Hanciv](https://discuss.elastic.co/u/Hanciv)
#### Post date: [July 26, 2018, 5:17pm UTC](https://discuss.elastic.co/t/logstash-s3-inout-plugin-crashes-when-reading-glacier-object/141822/1 "2018-07-26T17:17:00Z")

</div>

Hi,

I have configured logstash to read from AWS S3 bucket and send the data to Elastic Search. There is also a lifecycle policy setup in the S3 bucket to archive objects to glaicer after reaching 1 month. When I run logstash with S3 input plugin and if it encounters a glacier object, then it crahses and no longer push the data. Is there a way to skip glaicer objects or is there a workaround for this?

Thanks in advane for your help!!

Thanks

---

<div class="post-metadata">

### Author: ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)
#### Post date: [July 26, 2018, 10:38pm UTC](https://discuss.elastic.co/t/logstash-s3-inout-plugin-crashes-when-reading-glacier-object/141822/2 "2018-07-26T22:38:37Z")

</div>

How does it "crash"? Does all of Logstash crash, or is the pipeline merely restarted? Is there any helpful log output? When running with debug-level logging enabled, are there any backtraces in the logs? These would all be helpful.

You may want to open an issue on the [S3 Input Plugin](https://github.com/logstash-plugins/logstash-input-s3).

If you do, please be sure to include as clear reproduction steps as possible (setting up a public bucket that has one or more files in the states described would be super helpful).

---

<div class="post-metadata">

### Author: ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)
#### Post date: [August 8, 2018, 6:30pm UTC](https://discuss.elastic.co/t/logstash-s3-inout-plugin-crashes-when-reading-glacier-object/141822/3 "2018-08-08T18:30:03Z")

</div>

I've opened up a PR on the plugin to add support for skipping glacier-archived objects: [https://github.com/logstash-plugins/logstash-input-s3/pull/160](https://github.com/logstash-plugins/logstash-input-s3/pull/160)

---

<div class="post-metadata">

### Author: ![Hanciv](https://avatars.discourse-cdn.com/v4/letter/h/ea666f/32.png) [@Hanciv](https://discuss.elastic.co/u/Hanciv)
#### Post date: [August 9, 2018, 4:06pm UTC](https://discuss.elastic.co/t/logstash-s3-inout-plugin-crashes-when-reading-glacier-object/141822/4 "2018-08-09T16:06:56Z")

</div>

Hi,

I have a S3 bucket that has objects belonging to both Glacier and standard s3 storage class. When we start logstash with your S3 input listening to this bucket, the plugin fails with the below error and then onwards no events are being sent to Elastic Search

[2018-08-09T16:02:43,887][ERROR][logstash.pipeline] A plugin had an unrecoverable error. Will restart this plugin.  
Plugin: \<LogStash::Inputs::S3 bucket=\>"abcd", prefix=\>"input/", access\_key\_id=\>"XXXXXXXXXXXXXXXX", secret\_access\_key=\>"XXXXXXXXXXXXXXX", region=\>"us-east-1", temporary\_directory=\>"/home/logstash-5.4.1/tmp/logstash", id=\>"0475943184b1d0293ba2409b3baf36d958-1", enable\_metric=\>true, codec=\>\<LogStash::Codecs::Plain id=\>"plain\_41a047bc-9ea2-4a9b-b16c-a86790633cb3", enable\_metric=\>true, charset=\>"UTF-8"\>, delete=\>false, interval=\>60\>  
Error: The operation is not valid for the object's storage class

Thanks

---

<div class="post-metadata">

### Author: ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)
#### Post date: [August 16, 2018, 4:33am UTC](https://discuss.elastic.co/t/logstash-s3-inout-plugin-crashes-when-reading-glacier-object/141822/5 "2018-08-16T04:33:04Z")

</div>

Yes. The feature to skip s3 objects that gave been archived to glacier is currently in a pull-request that has not been merged and is therefore not yet available.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 13, 2018, 4:33am UTC](https://discuss.elastic.co/t/logstash-s3-inout-plugin-crashes-when-reading-glacier-object/141822/6 "2018-09-13T04:33:12Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
