Trying to upgrade Elasticsearch from 8.2 to 8.4.3 running on Windows. I copied the config folder from the old version to the new one, adjusted file paths in elasticsearch.yml, deleted the old service, then ran elasticsearch-service.bat install. When I try to run the service, I get a service start fail error. If I run elasticsearch from the command line, it works fine. Below are the contents of the elasticsearch logs when they fail...as far as I can tell, they don't say anything.
How can I troubleshoot this further, or do the below logs tell you guys something that I don't see??
Failed Service Run
[2022-10-05 22:22:13] [debug] ( prunsrv.c:1994) [ 2688] Apache Commons Daemon procrun log initialized.
[2022-10-05 22:22:13] [info] ( prunsrv.c:2002) [ 2688] Apache Commons Daemon procrun (1.3.1.0 64-bit) started.
[2022-10-05 22:22:13] [info] ( prunsrv.c:1906) [ 2688] Running Service 'elasticsearch-service-x64'...
[2022-10-05 22:22:13] [debug] ( prunsrv.c:1677) [ 1908] Inside serviceMain()...
[2022-10-05 22:22:13] [debug] ( prunsrv.c:1123) [ 1908] reportServiceStatusE: dwCurrentState = 2 (SERVICE_START_PENDING), dwWin32ExitCode = 0, dwWaitHint = 3000 milliseconds, dwServiceSpecificExitCode = 0.
[2022-10-05 22:22:13] [info] ( prunsrv.c:1422) [ 1908] Starting service...
[2022-10-05 22:22:13] [debug] ( prunsrv.c:497 ) [ 1908] Checking Java options for environment variable requirements
[2022-10-05 22:22:13] [debug] ( prunsrv.c:501 ) [ 1908] Checking environment variable requirements for '-XX:+UseSerialGC'
[2022-10-05 22:22:13] [debug] ( prunsrv.c:501 ) [ 1908] Checking environment variable requirements for '-Des.path.home=D:\ElasticStack\Apps\Elasticsearch\8.4.3'
[2022-10-05 22:22:13] [debug] ( prunsrv.c:501 ) [ 1908] Checking environment variable requirements for '-Des.path.conf=D:\ElasticStack\Apps\Elasticsearch\8.4.3\config'
[2022-10-05 22:22:13] [debug] ( prunsrv.c:501 ) [ 1908] Checking environment variable requirements for '-Des.distribution.type=zip'
[2022-10-05 22:22:13] [debug] ( prunsrv.c:501 ) [ 1908] Checking environment variable requirements for '-Dcli.name=windows-service-daemon'
[2022-10-05 22:22:13] [debug] ( prunsrv.c:501 ) [ 1908] Checking environment variable requirements for '-Dcli.libs=lib/tools/server-cli,lib/tools/windows-service-cli'
[2022-10-05 22:22:13] [debug] ( javajni.c:216 ) [ 1908] Explicit RuntimeLib specified 'D:\ElasticStack\Apps\Elasticsearch\8.4.3\jdk\bin\server\jvm.dll'
[2022-10-05 22:22:13] [debug] ( javajni.c:280 ) [ 1908] Adding Java bin path to the PATH to fix loading of awt.dll: 'D:\ElasticStack\Apps\Elasticsearch\8.4.3\jdk\bin'
[2022-10-05 22:22:13] [debug] ( javajni.c:286 ) [ 1908] Loading JVM DLL 'D:\ElasticStack\Apps\Elasticsearch\8.4.3\jdk\bin\server\jvm.dll'
[2022-10-05 22:22:13] [debug] ( javajni.c:414 ) [ 1908] Loaded JVM DLL 'D:\ElasticStack\Apps\Elasticsearch\8.4.3\jdk\bin\server\jvm.dll', home '(null)'.
[2022-10-05 22:22:13] [debug] ( javajni.c:419 ) [ 1908] JNI_GetCreatedJavaVMs...
[2022-10-05 22:22:13] [debug] ( javajni.c:817 ) [ 6208] JVM Option[0] -XX:+UseSerialGC
[2022-10-05 22:22:13] [debug] ( javajni.c:817 ) [ 6208] JVM Option[1] -Des.path.home=D:\ElasticStack\Apps\Elasticsearch\8.4.3
[2022-10-05 22:22:13] [debug] ( javajni.c:817 ) [ 6208] JVM Option[2] -Des.path.conf=D:\ElasticStack\Apps\Elasticsearch\8.4.3\config
[2022-10-05 22:22:13] [debug] ( javajni.c:817 ) [ 6208] JVM Option[3] -Des.distribution.type=zip
[2022-10-05 22:22:13] [debug] ( javajni.c:817 ) [ 6208] JVM Option[4] -Dcli.name=windows-service-daemon
[2022-10-05 22:22:13] [debug] ( javajni.c:817 ) [ 6208] JVM Option[5] -Dcli.libs=lib/tools/server-cli,lib/tools/windows-service-cli
[2022-10-05 22:22:13] [debug] ( javajni.c:817 ) [ 6208] JVM Option[6] -Djava.class.path=D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/ecs-logging-core-1.2.0.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/elasticsearch-8.4.3.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/elasticsearch-cli-8.4.3.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/elasticsearch-core-8.4.3.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/elasticsearch-geo-8.4.3.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/elasticsearch-log4j-8.4.3.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/elasticsearch-logging-8.4.3.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/elasticsearch-lz4-8.4.3.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/elasticsearch-plugin-classloader-8.4.3.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/elasticsearch-secure-sm-8.4.3.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/elasticsearch-x-content-8.4.3.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/HdrHistogram-2.1.9.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/hppc-0.8.1.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/jna-5.10.0.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/jopt-simple-5.0.2.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/log4j-api-2.18.0.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/log4j2-ecs-layout-1.2.0.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/lucene-analysis-common-9.3.0.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/lucene-backward-codecs-9.3.0.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/lucene-core-9.3.0.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/lucene-grouping-9.3.0.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/lucene-highlighter-9.3.0.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/lucene-join-9.3.0.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/lucene-memory-9.3.0.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/lucene-misc-9.3.0.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/lucene-queries-9.3.0.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/lucene-queryparser-9.3.0.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/lucene-sandbox-9.3.0.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/lucene-suggest-9.3.0.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/lz4-java-1.8.0.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/t-digest-3.2.jar;D:\ElasticStack\Apps\Elasticsearch\8.4.3/lib/cli-launcher/cli-launcher-8.4.3.jar
[2022-10-05 22:22:13] [debug] ( javajni.c:817 ) [ 6208] JVM Option[7] exit
[2022-10-05 22:22:13] [debug] ( javajni.c:817 ) [ 6208] JVM Option[8] abort
[2022-10-05 22:22:13] [debug] ( javajni.c:817 ) [ 6208] JVM Option[9] -Xms20480m
[2022-10-05 22:22:13] [debug] ( javajni.c:817 ) [ 6208] JVM Option[10] -Xmx20480m
[2022-10-05 22:22:13] [debug] ( javajni.c:817 ) [ 6208] JVM Option[11] -Xss1024k
[2022-10-05 22:22:13] [debug] ( javajni.c:1003) [ 6208] argv[0] = --quiet
[2022-10-05 22:22:13] [debug] ( javajni.c:1062) [ 6208] Java worker thread started for org/elasticsearch/launcher/CliToolLauncher:main
[2022-10-05 22:22:13] [debug] ( javajni.c:1068) [ 6208] JNI calling static void method org/elasticsearch/launcher/CliToolLauncher:main
[2022-10-05 22:22:14] [debug] ( javajni.c:566 ) [ 4996] Exit hook with exit code 64
[2022-10-05 22:22:14] [debug] ( prunsrv.c:1199) [ 4996] Start exit hook called...
[2022-10-05 22:22:14] [debug] ( prunsrv.c:1200) [ 4996] JVM exit code: 64.
Success Run From Command Line
[2022-10-05T22:25:42,920][INFO ][o.e.n.Node ] [ssselklog.contoso.net] version[8.4.3], pid[4980], build[zip/42f05b9372a9a4a470db3b52817899b99a76ee73/2022-10-04T07:17:24.662462378Z], OS[Windows Server 2012 R2/6.3/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/18.0.2.1/18.0.2.1+1-1]
[2022-10-05T22:25:42,922][INFO ][o.e.n.Node ] [ssselklog.contoso.net] JVM home [D:\ElasticStack\Apps\Elasticsearch\8.4.3\jdk], using bundled JDK [true]
[2022-10-05T22:25:42,923][INFO ][o.e.n.Node ] [ssselklog.contoso.net] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -Djava.security.manager=allow, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=ALL-UNNAMED, -Xms20g, -Xmx20g, -XX:+UseG1GC, -Djava.io.tmpdir=C:\Users\user\AppData\Local\Temp\elasticsearch, -XX:+HeapDumpOnOutOfMemoryError, -XX:+ExitOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=10737418240, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=25, -Des.distribution.type=zip, --module-path=D:\ElasticStack\Apps\Elasticsearch\8.4.3\lib, --add-modules=jdk.net, -Djdk.module.main=org.elasticsearch.server]