hi community
My elasticsearch Server is Down, ¿what could be happening?
Elasticsearch 8.7
1) systemctl status elasticsearch
elasticsearch.service - Elasticsearch
Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
Active: failed (Result: oom-kill) since Mon 2023-04-17 20:14:58 -05; 13min ago
Docs: https://www.elastic.co
Process: 72295 ExecStart=/usr/share/elasticsearch/bin/systemd-entrypoint -p ${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=143)
Main PID: 72295 (code=exited, status=143)
CPU: 19min 45.197s
Starting Elasticsearch...
Started Elasticsearch.
elasticsearch.service: A process of this unit has been killed by the OOM killer.
ERROR: Elasticsearch exited unexpectedly
elasticsearch.service: Failed with result 'oom-kill'.
elasticsearch.service: Consumed 19min 45.197s CPU time.
2) Logs /var/log/elasticsearch elasticsearch.log
root@XXX:/var/log/elasticsearch# tail -f elasticsearch.log
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1343) ~[?:?]
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1236) ~[?:?]
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1285) ~[?:?]
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:529) ~[?:?]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:468) ~[?:?]
... 16 more
[2023-04-17T20:14:45,952][WARN ][o.e.h.n.Netty4HttpServerTransport] [XXX] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/192.0.1.20:9200, remoteAddress=/192.0.1.20:59818}
[2023-04-17T20:14:47,674][WARN ][o.e.h.n.Netty4HttpServerTransport] [XXX] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/192.0.1.20:9200, remoteAddress=/192.0.1.20:59822}
[2023-04-17T20:14:48,398][WARN ][o.e.h.n.Netty4HttpServerTransport] [XXX] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/192.0.1.20:9200, remoteAddress=/192.0.1.20:59832}
[2023-04-17T20:14:50,570][WARN ][o.e.h.n.Netty4HttpServerTransport] [XXX] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/192.0.1.20:9200, remoteAddress=/192.0.1.20:59840}
- journalctl -xe
abr 17 20:39:28 XXX kernel: [ 73840] 1000 73840 3682 308 69632 1 0 sshd
abr 17 20:39:28 XXX kernel: [ 73841] 1000 73841 2042 359 49152 28 0 bash
abr 17 20:39:28 XXX kernel: [ 73847] 1000 73847 2502 146 65536 4 0 su
abr 17 20:39:28 XXX kernel: [ 73848] 0 73848 1831 150 49152 3 0 bash
abr 17 20:39:28 XXX kernel: [ 73849] 0 73849 32706 164 184320 6 0 journalctl
abr 17 20:39:28 XXX kernel: [ 73850] 0 73850 1461 56 57344 4 0 less
abr 17 20:39:28 XXX kernel: [ 73991] 0 73991 3630 307 73728 0 0 sshd
abr 17 20:39:28 XXX kernel: [ 73997] 1000 73997 3682 309 73728 0 0 sshd
abr 17 20:39:28 XXX kernel: [ 73998] 1000 73998 2042 386 57344 0 0 bash
abr 17 20:39:28 XXX kernel: [ 74001] 1000 74001 2502 149 57344 0 0 su
abr 17 20:39:28 XXX kernel: [ 74002] 0 74002 1819 149 53248 0 0 bash
abr 17 20:39:28 XXX kernel: [ 74025] 0 74025 2535 68 57344 0 0 systemctl
abr 17 20:39:28 XXX kernel: [ 74026] 0 74026 3294 149 65536 0 0 systemd-tty-ask
abr 17 20:39:28 XXX kernel: [ 74027] 107 74027 650623 591 430080 17051 0 java
abr 17 20:39:28 XXX kernel: [ 74086] 107 74086 1170357 470781 4648960 72371 0 java
abr 17 20:39:28 XXX kernel: [ 74112] 107 74112 22876 130 77824 0 0 controller
abr 17 20:39:28 XXX kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/system.slice/elasticsearch.service,task=java,pid=7408>
abr 17 20:39:28 XXX kernel: Out of memory: Killed process 74086 (java) total-vm:4681428kB, anon-rss:1883124kB, file-rss:0kB, shmem-rss:0kB, UID:107 pgtables:4540kB oom_score_adj:0
abr 17 20:39:28 XXX kibana[72442]: [2023-04-17T20:39:26.012-05:00][WARN ][plugins.licensing] License information could not be obtained from Elasticsearch due to ConnectionError: co>
abr 17 20:39:28 XXX kibana[72442]: [2023-04-17T20:39:26.642-05:00][WARN ][plugins.licensing] License information could not be obtained from Elasticsearch due to ConnectionError: co>
abr 17 20:39:30 XXX systemd-entrypoint[74027]: ERROR: Elasticsearch exited unexpectedly
abr 17 20:39:30 XXX systemd[1]: elasticsearch.service: Main process exited, code=exited, status=137/n/a
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: Debian -- User Support
░░
░░ An ExecStart= process belonging to unit elasticsearch.service has exited.
░░
░░ The process' exit code is 'exited' and its exit status is 137.
abr 17 20:39:30 XXX systemd[1]: elasticsearch.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: Debian -- User Support
░░
░░ The unit elasticsearch.service has entered the 'failed' state with result 'exit-code'.
abr 17 20:39:30 XXX systemd[1]: Failed to start Elasticsearch.
░░ Subject: A start job for unit elasticsearch.service has failed
░░ Defined-By: systemd
░░ Support: Debian -- User Support
░░
░░ A start job for unit elasticsearch.service has finished with a failure.
░░
░░ The job identifier is 11055 and the job result is failed.
abr 17 20:39:30 XXX systemd[1]: elasticsearch.service: Consumed 55.067s CPU time.
░░ Subject: Resources consumed by unit runtime
░░ Defined-By: systemd
░░ Support: Debian -- User Support
░░
░░ The unit elasticsearch.service completed and consumed the indicated resources.
4) Please help me ...