# Elasticsearch failed to delete unreferenced translog files due to unexpected Python and OpenSSL upgrade. If the issue is related to upgrade it should be infrastructure independent

**URL:** <https://discuss.elastic.co/t/elasticsearch-failed-to-delete-unreferenced-translog-files-due-to-unexpected-python-and-openssl-upgrade-if-the-issue-is-related-to-upgrade-it-should-be-infrastructure-independent/109168>\
**Category:** Elasticsearch\
**Created:** [November 27, 2017, 5:33am UTC](https://discuss.elastic.co/t/elasticsearch-failed-to-delete-unreferenced-translog-files-due-to-unexpected-python-and-openssl-upgrade-if-the-issue-is-related-to-upgrade-it-should-be-infrastructure-independent/109168 "2017-11-27T05:33:51Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![david\_loh](https://avatars.discourse-cdn.com/v4/letter/d/a4c791/32.png) [@david\_loh](https://discuss.elastic.co/u/david_loh)\
**Post date:** [November 27, 2017, 5:33am UTC](https://discuss.elastic.co/t/elasticsearch-failed-to-delete-unreferenced-translog-files-due-to-unexpected-python-and-openssl-upgrade-if-the-issue-is-related-to-upgrade-it-should-be-infrastructure-independent/109168/1 "2017-11-27T05:33:51Z")

</div>

Hi all,

I got a production Elasticsearch cluster running on Azure.

On 9th Nov, Azure agent started a yum update on OpenSSL and Python libraries. This action crashes one of the index.

Have anyone with Elasticsearch on Azure experience this before ?

I would like to know if Elasticsearch is indexing data and the following libraries has been updated, Will it crash Elasticsearch ?

(Extract from /var/log/yum.log)

Nov 09 10:39:14 Updated: 1:openssl-libs-1.0.2k-8.el7.x86\_64  
Nov 09 10:39:17 Updated: python-libs-2.7.5-58.el7.x86\_64  
Nov 09 10:39:18 Updated: openssh-7.4p1-13.el7\_4.x86\_64  
Nov 09 10:39:18 Updated: python-2.7.5-58.el7.x86\_64  
Nov 09 10:39:18 Updated: libcurl-7.29.0-42.el7.x86\_64  
Nov 09 10:39:18 Updated: curl-7.29.0-42.el7.x86\_64  
Nov 09 10:39:19 Updated: python-devel-2.7.5-58.el7.x86\_64  
Nov 09 10:39:19 Updated: openssh-clients-7.4p1-13.el7\_4.x86\_64  
Nov 09 10:39:20 Updated: openssh-server-7.4p1-13.el7\_4.x86\_64  
Nov 09 10:39:21 Updated: 1:openssl-1.0.2k-8.el7.x86\_64  
Nov 09 10:39:22 Updated: 1:openssl-devel-1.0.2k-8.el7.x86\_64  
Nov 09 10:39:23 Updated: wget-1.14-15.el7\_4.1.x86\_64

(extract from /var/log/messages)

Nov 9 10:37:45 es0 python: 2017/11/09 10:37:45.027369 INFO Event: name=WALinuxAgent, op=HeartBeat, message=, duration=0  
Nov 9 10:38:33 es0 python: 2017/11/09 10:38:33.252548 INFO [Microsoft.Azure.Extensions.CustomScript-2.0.4] Target handler state: enabled  
Nov 9 10:38:33 es0 python: 2017/11/09 10:38:33.254394 INFO [Microsoft.Azure.Extensions.CustomScript-2.0.4] [Enable] current handler state is: notinstalled  
Nov 9 10:38:33 es0 python: 2017/11/09 10:38:33.434750 INFO Event: name=Microsoft.Azure.Extensions.CustomScript, op=Download, message=Download succeeded, duration=0  
Nov 9 10:38:33 es0 python: 2017/11/09 10:38:33.435728 INFO [Microsoft.Azure.Extensions.CustomScript-2.0.4] Initialize extension directory  
Nov 9 10:38:33 es0 python: 2017/11/09 10:38:33.440453 INFO [Microsoft.Azure.Extensions.CustomScript-2.0.4] Update settings file: 0.settings  
Nov 9 10:38:33 es0 python: 2017/11/09 10:38:33.441408 INFO [Microsoft.Azure.Extensions.CustomScript-2.0.4] Disable extension [bin/custom-script-shim disable]  
Nov 9 10:38:34 es0 python: 2017/11/09 10:38:34.446451 INFO Event: name=Microsoft.Azure.Extensions.CustomScript, op=Disable, message=Launch command succeeded: bin/custom-script-shim disable, duration=0  
Nov 9 10:38:34 es0 python: 2017/11/09 10:38:34.448416 INFO [Microsoft.Azure.Extensions.CustomScript-2.0.4] Copy status files from old plugin to new  
Nov 9 10:38:34 es0 python: 2017/11/09 10:38:34.448956 INFO [Microsoft.Azure.Extensions.CustomScript-2.0.4] Update extension [bin/custom-script-shim update]  
Nov 9 10:38:35 es0 python: 2017/11/09 10:38:35.460504 INFO Event: name=Microsoft.Azure.Extensions.CustomScript, op=Update, message=Launch command succeeded: bin/custom-script-shim update, duration=0  
Nov 9 10:38:35 es0 python: 2017/11/09 10:38:35.462176 INFO [Microsoft.Azure.Extensions.CustomScript-2.0.4] Uninstall extension [bin/custom-script-shim uninstall]  
Nov 9 10:38:36 es0 python: 2017/11/09 10:38:36.480447 INFO Event: name=Microsoft.Azure.Extensions.CustomScript, op=UnInstall, message=Launch command succeeded: bin/custom-script-shim uninstall, duration=0  
Nov 9 10:38:36 es0 python: 2017/11/09 10:38:36.481668 INFO [Microsoft.Azure.Extensions.CustomScript-2.0.4] Remove extension handler directory: /var/lib/waagent/Microsoft.Azure.Extensions.CustomScript-2.0.4  
Nov 9 10:38:36 es0 python: 2017/11/09 10:38:36.483629 INFO [Microsoft.Azure.Extensions.CustomScript-2.0.4] Install extension [bin/custom-script-shim install]  
Nov 9 10:38:37 es0 python: 2017/11/09 10:38:37.524445 INFO Event: name=Microsoft.Azure.Extensions.CustomScript, op=Install, message=Launch command succeeded: bin/custom-script-shim install, duration=0  
Nov 9 10:38:37 es0 python: 2017/11/09 10:38:37.530731 INFO [Microsoft.Azure.Extensions.CustomScript-2.0.4] Enable extension [bin/custom-script-shim enable]  
Nov 9 10:38:38 es0 python: 2017/11/09 10:38:38.556536 INFO Event: name=Microsoft.Azure.Extensions.CustomScript, op=Enable, message=Launch command succeeded: bin/custom-script-shim enable, duration=0  
Nov 9 10:38:39 es0 kernel: blk\_update\_request: I/O error, dev fd0, sector 0  
Nov 9 10:38:39 es0 elasticsearch: [2017-11-09 10:38:39,253][WARN][index.translog] [es0.----.prod] [xxxx-yyy-7-1\_index\_2017.11.09][1] failed to delete unreferenced translog files  
Nov 9 10:38:39 es0 elasticsearch: java.nio.file.NoSuchFileException: /data3/xxx/nodes/0/indices/xxxx-yyyy-7-1\_index\_2017.11.09/1/translog  
Nov 9 10:38:39 es0 elasticsearch: at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)  
Nov 9 10:38:39 es0 elasticsearch: at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)  
Nov 9 10:38:39 es0 elasticsearch: at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)  
Nov 9 10:38:39 es0 elasticsearch: at sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:427)  
Nov 9 10:38:39 es0 elasticsearch: at java.nio.file.Files.newDirectoryStream(Files.java:457)

Regards,  
David

---

<div class="post-metadata">

**Author:** ![jasontedor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jasontedor/32/66992_2.png) [@jasontedor](https://discuss.elastic.co/u/jasontedor)\
**Post date:** [November 28, 2017, 2:26am UTC](https://discuss.elastic.co/t/elasticsearch-failed-to-delete-unreferenced-translog-files-due-to-unexpected-python-and-openssl-upgrade-if-the-issue-is-related-to-upgrade-it-should-be-infrastructure-independent/109168/2 "2017-11-28T02:26:51Z")

</div>

This has nothing to do with the upgrades.

> [@david\_loh](#):
>
> Nov 9 10:38:39 es0 kernel: blk\_update\_request: I/O error, dev fd0, sector 0

This is the problem, the hardware is failing.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 26, 2017, 2:27am UTC](https://discuss.elastic.co/t/elasticsearch-failed-to-delete-unreferenced-translog-files-due-to-unexpected-python-and-openssl-upgrade-if-the-issue-is-related-to-upgrade-it-should-be-infrastructure-independent/109168/3 "2017-12-26T02:27:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
