# NullPointerException when getting list of snapshots on S3

**URL:** <https://discuss.elastic.co/t/nullpointerexception-when-getting-list-of-snapshots-on-s3/94458>\
**Category:** Elasticsearch\
**Created:** [July 25, 2017, 9:57am UTC](https://discuss.elastic.co/t/nullpointerexception-when-getting-list-of-snapshots-on-s3/94458 "2017-07-25T09:57:17Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![eirc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eirc/32/11769_2.png) [@eirc](https://discuss.elastic.co/u/eirc)\
**Post date:** [July 25, 2017, 9:57am UTC](https://discuss.elastic.co/t/nullpointerexception-when-getting-list-of-snapshots-on-s3/94458/1 "2017-07-25T09:57:17Z")

</div>

Hello,

I'm using the repository-s3 plugin and after using it for some months I now cannot get the list of snapshots:

```
curl elasticsearch:9200/_snapshot/long_term/_all
{"error":{"root_cause":[{"type":"remote_transport_exception","reason":"[SVVyQPF][10.127.1.203:9300][cluster:admin/snapshot/get]"}],"type":"null_pointer_exception","reason":null},"status":500}

```

Elasticsearch logs this:

```
[2017-07-25T12:01:47,038][WARN][r.suppressed] path: /_snapshot/long_term/_all, params: {repository=long_term, snapshot=_all}
org.elasticsearch.transport.RemoteTransportException: [SVVyQPF][10.127.1.203:9300][cluster:admin/snapshot/get]
Caused by: java.lang.NullPointerException

```

I use curator to take the backups and after grabbing backups successfully it fails when it tries to delete old snapshots because that's when it requires a list too:

```
2017-07-25 11:53:02,191 ERROR Failed to complete action: delete_snapshots. <class 'curator.exceptions.FailedExecution'>: Unable to get snapshot information from repository: long_term. Error: TransportError(500, 'null_pointer_exception', '[SVVyQPF][10.127.1.203:9300][cluster:admin/snapshot/get]')

```

I have a feeling this is due to some kind of timeout. I turned on debug logging and while I couldn't find a more specific reason this fails I noticed it made ~ 2K requests to S3 until it failed and it stopped at 90 seconds. Is this a configurable timeout?

Please help! I got no access to my backups!

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [July 25, 2017, 10:10am UTC](https://discuss.elastic.co/t/nullpointerexception-when-getting-list-of-snapshots-on-s3/94458/2 "2017-07-25T10:10:27Z")

</div>

That sounds very bad.

I think you should open an issue in elasticsearch.  
Can you tell more about your version?

---

<div class="post-metadata">

**Author:** ![eirc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eirc/32/11769_2.png) [@eirc](https://discuss.elastic.co/u/eirc)\
**Post date:** [July 25, 2017, 10:11am UTC](https://discuss.elastic.co/t/nullpointerexception-when-getting-list-of-snapshots-on-s3/94458/3 "2017-07-25T10:11:01Z")

</div>

Just upgraded to 5.5.0.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [July 25, 2017, 10:11am UTC](https://discuss.elastic.co/t/nullpointerexception-when-getting-list-of-snapshots-on-s3/94458/4 "2017-07-25T10:11:33Z")

</div>

And it appeared just after the upgrade?

---

<div class="post-metadata">

**Author:** ![eirc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eirc/32/11769_2.png) [@eirc](https://discuss.elastic.co/u/eirc)\
**Post date:** [July 25, 2017, 10:18am UTC](https://discuss.elastic.co/t/nullpointerexception-when-getting-list-of-snapshots-on-s3/94458/5 "2017-07-25T10:18:58Z")

</div>

Nope it started a few days before so it started while on 5.4.1.`curl`ing for the list of snapshots took increasingly long in the past but it still worked. Eventually I reached a curator timeout which I increased and it was working again. Now it seems to be breaking before it hits that timeout.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [July 25, 2017, 10:25am UTC](https://discuss.elastic.co/t/nullpointerexception-when-getting-list-of-snapshots-on-s3/94458/6 "2017-07-25T10:25:38Z")

</div>

I see. AFAIK we don't set any 90s timeout when we are using the AWS SDK and defaults don't seem to set that as per [http://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/section-client-configuration.html](http://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/section-client-configuration.html).

We are just setting the setSocketTimeout which is by default `50s`.

> <https://github.com/elastic/elasticsearch/blob/5.5/plugins/repository-s3/src/main/java/org/elasticsearch/repositories/s3/S3ClientSettings.java#L81-L82>

And AFAIK it only set the timeout for a single API call.

May be it's a more global timeout that we have in snapshot/restore? @Igor_Motov Do you have an idea?

---

<div class="post-metadata">

**Author:** ![eirc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eirc/32/11769_2.png) [@eirc](https://discuss.elastic.co/u/eirc)\
**Post date:** [July 25, 2017, 10:27am UTC](https://discuss.elastic.co/t/nullpointerexception-when-getting-list-of-snapshots-on-s3/94458/7 "2017-07-25T10:27:31Z")

</div>

Opened an issue too: [https://github.com/elastic/elasticsearch/issues/25878](https://github.com/elastic/elasticsearch/issues/25878)

---

<div class="post-metadata">

**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\
**Post date:** [July 25, 2017, 1:28pm UTC](https://discuss.elastic.co/t/nullpointerexception-when-getting-list-of-snapshots-on-s3/94458/8 "2017-07-25T13:28:27Z")

</div>

Can you check logs on `10.127.1.203` around `2017-07-25T12:01:47` to see if there are any other messages there?

---

<div class="post-metadata">

**Author:** ![eirc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eirc/32/11769_2.png) [@eirc](https://discuss.elastic.co/u/eirc)\
**Post date:** [July 25, 2017, 2:10pm UTC](https://discuss.elastic.co/t/nullpointerexception-when-getting-list-of-snapshots-on-s3/94458/9 "2017-07-25T14:10:25Z")

</div>

Added some more logs to the issue [https://github.com/elastic/elasticsearch/issues/25878#issuecomment-317748498](https://github.com/elastic/elasticsearch/issues/25878#issuecomment-317748498). Regarding the time I am providing logs from a request initiated at `2017-07-25T12:26:30` because that's when I had DEBUG on.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 22, 2017, 2:10pm UTC](https://discuss.elastic.co/t/nullpointerexception-when-getting-list-of-snapshots-on-s3/94458/10 "2017-08-22T14:10:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
