Elasticsearch running as service on Ubuntu is failing

I am trying to run elastisearch as a service on an ubuntu server but it keeps failing and I can't figure out what is wrong. Never installed elasticsearch on an ubuntu server. Version is 6.7.1

P.S. server.com isnt the name of the server just for security purposes.

ā— elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Wed 2019-06-12 15:55:12 EDT; 6s ago
Docs: http://www.elastic.co
Process: 3889 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=1/FAILURE)
Main PID: 3889 (code=exited, status=1/FAILURE)

Jun 12 15:55:12 server.com systemd[1]: Started Elasticsearch.
Jun 12 15:55:12 server.com systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE
Jun 12 15:55:12 server.com systemd[1]: elasticsearch.service: Unit entered failed state.
Jun 12 15:55:12 server.com systemd[1]: elasticsearch.service: Failed with result 'exit-code'.

What do your actual Elasticsearch logs show?

Java HotSpot(TM) 64-Bit Server VM (25.212-b25) for linux-amd64 JRE (1.8.0_212-b25), built on Apr 3 2019 01:21:17 by "fedora" with gcc 7.3.0
Memory: 4k page, physical 16414324k(15081116k free), swap 7999484k(7999484k free)
CommandLine flags: -XX:+AlwaysPreTouch -XX:CMSInitiatingOccupancyFraction=75 -XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log -XX:GCLogFileSize=67108864 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/lib/elasticsearch -XX:InitialHeapSize=1073741824 -XX:MaxHeapSize=1073741824 -XX:MaxNewSize=357916672 -XX:MaxTenuringThreshold=6 -XX:NewSize=357916672 -XX:NumberOfGCLogFiles=32 -XX:OldPLABSize=16 -XX:OldSize=715825152 -XX:-OmitStackTraceInFastThrow -XX:+PrintGC -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -XX:ThreadStackSize=1024 -XX:+UseCMSInitiatingOccupancyOnly -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC -XX:+UseGCLogFileRotation -XX:+UseParNewGC
2019-06-12T17:05:08.254-0400: 0.956: Total time for which application threads were stopped: 0.0001153 seconds, Stopping threads took: 0.0000515 seconds
2019-06-12T17:05:08.752-0400: 1.454: Total time for which application threads were stopped: 0.0001096 seconds, Stopping threads took: 0.0000321 seconds
2019-06-12T17:05:08.863-0400: 1.565: Total time for which application threads were stopped: 0.0001020 seconds, Stopping threads took: 0.0000353 seconds
2019-06-12T17:05:09.036-0400: 1.738: Total time for which application threads were stopped: 0.0001421 seconds, Stopping threads took: 0.0000315 seconds
2019-06-12T17:05:10.004-0400: 2.706: Total time for which application threads were stopped: 0.0002291 seconds, Stopping threads took: 0.0000401 seconds
2019-06-12T17:05:10.005-0400: 2.707: Total time for which application threads were stopped: 0.0000891 seconds, Stopping threads took: 0.0000257 seconds
2019-06-12T17:05:10.126-0400: 2.828: Total time for which application threads were stopped: 0.0001742 seconds, Stopping threads took: 0.0000333 seconds
2019-06-12T17:05:10.504-0400: 3.206: Total time for which application threads were stopped: 0.0001853 seconds, Stopping threads took: 0.0000359 seconds
2019-06-12T17:05:11.300-0400: 4.002: Total time for which application threads were stopped: 0.0002929 seconds, Stopping threads took: 0.0000503 seconds
2019-06-12T17:05:11.709-0400: 4.411: Total time for which application threads were stopped: 0.0002584 seconds, Stopping threads took: 0.0000356 seconds
2019-06-12T17:05:11.837-0400: 4.540: Total time for which application threads were stopped: 0.0002854 seconds, Stopping threads took: 0.0000647 seconds
2019-06-12T17:05:11.950-0400: 4.652: Total time for which application threads were stopped: 0.0002640 seconds, Stopping threads took: 0.0000317 seconds
2019-06-12T17:05:11.988-0400: 4.690: Total time for which application threads were stopped: 0.0002092 seconds, Stopping threads took: 0.0000291 seconds
2019-06-12T17:05:12.018-0400: 4.720: Total time for which application threads were stopped: 0.0001035 seconds, Stopping threads took: 0.0000723 seconds
2019-06-12T17:05:12.018-0400: 4.720: Total time for which application threads were stopped: 0.0000763 seconds, Stopping threads took: 0.0000406 seconds
2019-06-12T17:05:12.018-0400: 4.720: Total time for which application threads were stopped: 0.0000727 seconds, Stopping threads took: 0.0000467 seconds
2019-06-12T17:05:12.018-0400: 4.720: Total time for which application threads were stopped: 0.0000737 seconds, Stopping threads took: 0.0000462 seconds
2019-06-12T17:05:12.019-0400: 4.721: Total time for which application threads were stopped: 0.0001295 seconds, Stopping threads took: 0.0001025 seconds
Heap
par new generation total 314560K, used 95093K [0x00000000c0000000, 0x00000000d5550000, 0x00000000d5550000)
eden space 279616K, 34% used [0x00000000c0000000, 0x00000000c5cdd690, 0x00000000d1110000)
from space 34944K, 0% used [0x00000000d1110000, 0x00000000d1110000, 0x00000000d3330000)
to space 34944K, 0% used [0x00000000d3330000, 0x00000000d3330000, 0x00000000d5550000)
concurrent mark-sweep generation total 699072K, used 0K [0x00000000d5550000, 0x0000000100000000, 0x0000000100000000)
Metaspace used 16843K, capacity 17358K, committed 17664K, reserved 1064960K
class space used 2020K, capacity 2201K, committed 2304K, reserved 1048576K

This is what I get from the gc.log.0.current file

Please format your code/logs/config using the </> button, or markdown style back ticks. It helps to make things easy to read which helps us help you :slight_smile:

What is in /var/log/elasticsearch/elasticsearch.log ?

Sorry about that. And the only thing in that folder is the gc.log.0.current file there is no elasticsearch.log file

What about systemctl status elasticsearch?

When I do that I get the error message I included in the first message

I had the same problem and fixed it by setting PID_DIR in the yml file.

I got it running but it crashes in 30 seconds or less. Cannot figure it out yet

Also, when attempting to run it not as a service. It says gives a warning 'Failing back to Java on path. This behavior is deprecated. Specify JAVA_HOME.'

Ok how did you install things?

If i create the elasticsearch directory under /var/run/ should the elasticsearch user own it or root? Also should i create the elasticsearch.pid file? Or will that automatically be created by itself?

How did you install things - deb via apt?

I assume it was. I didn't install it and the person who did is unreachable at the moment. How could I figure this out, if necessary?

If you're on Ubuntu then check /etc/apt/sources.list.d/ for any mention of elastic.
Otherwise check dpkg -l|grep elasticsearch.

I checked the sources.list.d folder but did not see anything related to elastic. For the second command I got this:

ii elasticsearch-oss 6.7.1 all Elasticsearch is a distributed RESTful search engine built for the cloud. Reference documentation can be found at https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html and the 'Elasticsearch: The Definitive Guide' book can be found at https://www.elastic.co/guide/en/elasticsearch/guide/current/index.html

Please don't post pictures of text, they are difficult to read, impossible to search and some people may not be even able to see them :slight_smile:

Ok I changed it and it was through dpkg that elasticsearch was installed.

Any other information I can provide to maybe to shorten the list of possible sources of the issue?