Hi,
I'm doing an upgrade from ES 5.6 to ES 6.8. When starting ES 6.8, I'm getting this exception:
-- Logs begin at Tue 2020-10-20 00:29:07 UTC, end at Tue 2020-10-20 17:35:18 UTC. --
Oct 20 00:29:56 systemd[1]: Started Elasticsearch.
Oct 20 00:30:02 systemd[1]: elasticsearch.service: main process exited, code=exited, status=78/n/a
Oct 20 00:30:02 systemd[1]: Unit elasticsearch.service entered failed state.
Oct 20 00:30:02 systemd[1]: elasticsearch.service failed.
Oct 20 06:03:54 systemd[1]: Started Elasticsearch.
Oct 20 06:03:55 elasticsearch[117234]: ERROR: Temporary file directory [/var/lib/elasticsearch/tmp] does not exist or is not accessible
Oct 20 06:03:55 systemd[1]: elasticsearch.service: main process exited, code=exited, status=78/n/a
Oct 20 06:03:55 systemd[1]: Unit elasticsearch.service entered failed state.
Oct 20 06:03:55 systemd[1]: elasticsearch.service failed.
Oct 20 06:07:43 systemd[1]: Started Elasticsearch.
Oct 20 06:07:45 elasticsearch[118278]: 2020-10-20 06:07:45,919 main ERROR Unable to locate appender "deprecated_audit_rolling" for logger config "org.elasticsearch.xpack.security.audit.logfile.DeprecatedLoggingAuditTrail"
Oct 20 06:07:46 elasticsearch[118278]: Oct 20, 2020 6:07:46 AM com.sun.jna.Native loadNativeDispatchLibrary
Oct 20 06:07:46 elasticsearch[118278]: WARNING: JNA Warning: IOException removing temporary files
Oct 20 06:07:46 elasticsearch[118278]: java.io.IOException: JNA temporary directory '/var/lib/elasticsearch/tmp' is not writable
Oct 20 06:07:46 elasticsearch[118278]: at com.sun.jna.Native.getTempDir(Native.java:1310)
Oct 20 06:07:46 elasticsearch[118278]: at com.sun.jna.Native.removeTemporaryFiles(Native.java:1317)
Oct 20 06:07:46 elasticsearch[118278]: at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:919)
Oct 20 06:07:46 elasticsearch[118278]: at com.sun.jna.Native.<clinit>(Native.java:195)
Oct 20 06:07:46 elasticsearch[118278]: at java.lang.Class.forName0(Native Method)
Oct 20 06:07:46 elasticsearch[118278]: at java.lang.Class.forName(Class.java:264)
Oct 20 06:07:46 elasticsearch[118278]: at org.elasticsearch.bootstrap.Natives.<clinit>(Natives.java:45)
Oct 20 06:07:46 elasticsearch[118278]: at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:102)
Oct 20 06:07:46 elasticsearch[118278]: at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:170)
Oct 20 06:07:46 elasticsearch[118278]: at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333)
Oct 20 06:07:46 elasticsearch[118278]: at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159)
Oct 20 06:07:46 elasticsearch[118278]: at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150)
Oct 20 06:07:46 elasticsearch[118278]: at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
Oct 20 06:07:46 elasticsearch[118278]: at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
Oct 20 06:07:46 elasticsearch[118278]: at org.elasticsearch.cli.Command.main(Command.java:90)
Oct 20 06:07:46 elasticsearch[118278]: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116)
Oct 20 06:07:46 elasticsearch[118278]: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93)
Oct 20 06:07:46 elasticsearch[118278]: [2020-10-20T06:07:46,388][WARN ][o.e.b.Natives ] [unknown] unable to load JNA native support library, native methods will be disabled.
Oct 20 06:07:46 elasticsearch[118278]: java.lang.UnsatisfiedLinkError: Failed to create temporary file for /com/sun/jna/linux-x86-64/libjnidispatch.so library: JNA temporary directory '/var/lib/elasticsearch/tmp' is not writable
Oct 20 06:07:46 elasticsearch[118278]: at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:1032) ~[jna-5.5.0.jar:5.5.0 (b0)]
Oct 20 06:07:46 elasticsearch[118278]: at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:988) ~[jna-5.5.0.jar:5.5.0 (b0)]
Oct 20 06:07:46 elasticsearch[118278]: at com.sun.jna.Native.<clinit>(Native.java:195) ~[jna-5.5.0.jar:5.5.0 (b0)]
Oct 20 06:07:46 elasticsearch[118278]: at java.lang.Class.forName0(Native Method) ~[?:1.8.0_262]
Oct 20 06:07:46 elasticsearch[118278]: at java.lang.Class.forName(Class.java:264) ~[?:1.8.0_262]
Oct 20 06:07:46 elasticsearch[118278]: at org.elasticsearch.bootstrap.Natives.<clinit>(Natives.java:45) [elasticsearch-6.8.12.jar:6.8.12]
Oct 20 06:07:46 elasticsearch[118278]: at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:102) [elasticsearch-6.8.12.jar:6.8.12]
Oct 20 06:07:46 elasticsearch[118278]: at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:170) [elasticsearch-6.8.12.jar:6.8.12]
Oct 20 06:07:46 elasticsearch[118278]: at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) [elasticsearch-6.8.12.jar:6.8.12]
...
Oct 20 06:07:46 elasticsearch[118278]: ... 6 more
Oct 20 06:07:46 elasticsearch[118278]: Caused by: java.nio.file.AccessDeniedException: /var/lib/elasticsearch/tmp
Oct 20 06:07:46 elasticsearch[118278]: at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) ~[?:1.8.0_262]
Oct 20 06:07:46 elasticsearch[118278]: at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:1.8.0_262]
Oct 20 06:07:46 elasticsearch[118278]: at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:1.8.0_262]
Oct 20 06:07:46 elasticsearch[118278]: at sun.nio.fs.UnixFileSystemProvider.checkAccess(UnixFileSystemProvider.java:308) ~[?:1.8.0_262]
Oct 20 06:07:46 elasticsearch[118278]: at org.elasticsearch.bootstrap.Security.ensureDirectoryExists(Security.java:409) ~[elasticsearch-6.8.12.jar:6.8.12]
Oct 20 06:07:46 elasticsearch[118278]: at org.elasticsearch.bootstrap.FilePermissionUtils.addDirectoryPath(FilePermissionUtils.java:68) ~[elasticsearch-6.8.12.jar:6.8.12]
Oct 20 06:07:46 elasticsearch[118278]: at org.elasticsearch.bootstrap.Security.addFilePermissions(Security.java:291) ~[elasticsearch-6.8.12.jar:6.8.12]
Oct 20 06:07:46 elasticsearch[118278]: at org.elasticsearch.bootstrap.Security.createPermissions(Security.java:254) ~[elasticsearch-6.8.12.jar:6.8.12]
Oct 20 06:07:46 elasticsearch[118278]: at org.elasticsearch.bootstrap.Security.configure(Security.java:123) ~[elasticsearch-6.8.12.jar:6.8.12]
Oct 20 06:07:46 elasticsearch[118278]: at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:207) ~[elasticsearch-6.8.12.jar:6.8.12]
Oct 20 06:07:46 elasticsearch[118278]: at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.8.12.jar:6.8.12]
Oct 20 06:07:46 elasticsearch[118278]: at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.8.12.jar:6.8.12]
Oct 20 06:07:46 elasticsearch[118278]: ... 6 more
Oct 20 06:07:46 systemd[1]: elasticsearch.service: main process exited, code=exited, status=1/FAILURE
Oct 20 06:07:46 systemd[1]: Unit elasticsearch.service entered failed state.
Oct 20 06:07:46 systemd[1]: elasticsearch.service failed.
If I create the /var/lib/elasticsearch/tmp directory and give it the needed permissions, ES starts fine. But I didn't have to do this in 5.6 (in 5.6, either /var/lib/elasticsearch/tmp was automatically created or ES didn't try to write to it). Is this now a necessary preparation prior to starting ES 6.8?
Thanks,
Chris