Elasticsearch run with single noe but not running with docker-compose for multi-node

Elastic Stack Elasticsearch
1

Hi, I'm having issue running multi-node cluster using docker-compose on Apple M4 chip.

I was getting similar issue when I tried to run single node using docker but it was fixed by using

ES_JAVA_OPTS='-XX:UseSVE=0'
CLI_JAVA_OPTS='-XX:UseSVE=0'

Image used - docker.elastic.co/elasticsearch/elasticsearch:8.17.1

I tried some in yml file but It's not working

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGILL (0x4) at pc=0x0000ffff83d40028, pid=9, tid=18
#
# JRE version:  (23.0+37) (build )
# Java VM: OpenJDK 64-Bit Server VM (23+37-2369, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, serial gc, linux-aarch64)
# Problematic frame:
# j  java.lang.System.registerNatives()V+0 java.base@23
#
# No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

---------------  S U M M A R Y ------------

Command Line: -Xms4m -Xmx64m -XX:+UseSerialGC -Dcli.name= -Dcli.script=bin/elasticsearch-certutil -Dcli.libs=modules/x-pack-core,modules/x-pack-security,lib/tools/security-cli -Des.path.home=/usr/share/elasticsearch -Des.path.conf=/usr/share/elasticsearch/config -Des.distribution.type=docker org.elasticsearch.launcher.CliToolLauncher ca --silent --pem -out config/certs/ca.zip

Host: AArch64, 10 cores, 7G, Ubuntu 20.04.6 LTS
Time: Sat Feb  8 07:26:01 2025 UTC elapsed time: 0.006926 seconds (0d 0h 0m 0s)

---------------  T H R E A D  ---------------

Current thread (0x0000ffff9402a290):  JavaThread "Unknown thread" [_thread_in_native, id=18, stack(0x0000ffff9a9a5000,0x0000ffff9aba3000) (2040K)]

Stack: [0x0000ffff9a9a5000,0x0000ffff9aba3000],  sp=0x0000ffff9aba10b0,  free space=2032k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
j  java.lang.System.registerNatives()V+0 java.base@23
j  java.lang.System.<clinit>()V+0 java.base@23
v  ~StubRoutines::call_stub 0x0000ffff83d37118
V  [libjvm.so+0x7d7368]  JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, JavaThread*)+0x218
V  [libjvm.so+0x7b18f4]  InstanceKlass::call_class_initializer(JavaThread*)+0x294
V  [libjvm.so+0x7b280c]  InstanceKlass::initialize_impl(JavaThread*)+0x65c
V  [libjvm.so+0xcf7d24]  Threads::initialize_java_lang_classes(JavaThread*, JavaThread*)+0xe0
V  [libjvm.so+0xcf9cd4]  Threads::create_vm(JavaVMInitArgs*, bool*)+0x3d4
V  [libjvm.so+0x871f5c]  JNI_CreateJavaVM+0x7c
C  [libjli.so+0x46d0]  JavaMain+0x80
C  [libjli.so+0x7b2c]  ThreadJavaMain+0xc
C  [libpthread.so.0+0x7624]  start_thread+0x184
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  java.lang.System.registerNatives()V+0 java.base@23
j  java.lang.System.<clinit>()V+0 java.base@23
v  ~StubRoutines::call_stub 0x0000ffff83d37118

siginfo: si_signo: 4 (SIGILL), si_code: 1 (ILL_ILLOPC), si_addr: 0x0000ffff83d40028

Registers:
R0=0x0000000000000000
R1=0x0000000000000000
R2=0x0000000000000000
R3=0x0000000000000000
R4=0x000000007f30ffff
R5=0x000000000000ffff
R6=0x0000ffff9bed85c0
R7=0x000000000000000d
R8=0x0000000000000000
R9=0x000000000000000c
R10=0x000000000000000d
R11=0x00000004004b4ab0
R12=0x000000040017cc68
R13=0x000000040073aa50
R14=0x0000000000000000
R15=0x0000000000000000
R16=0x0000ffff9bec5858
R17=0x0000ffff9c008b00
R18=0x000000040017cb50
R19=0x0000ffff83d3dc30
R20=0x0000ffff9aba10b0
R21=0x0000ffff9bf74aa0
R22=0x0000000000000000
R23=0x0000ffff83d3eec0
R24=0x0000ffff9aba1138
R25=0x0000ffff9aba1330
R26=0x0000000400165758
R27=0x0000000000000000
R28=0x0000ffff9402a290
R29=0x0000ffff9aba1120
R30=0x0000ffff83d4000c


Register to memory mapping:

R0 =0x0 is null
R1 =0x0 is null
R2 =0x0 is null
R3 =0x0 is null
R4 =0x000000007f30ffff is an unknown value
R5 =0x000000000000ffff is an unknown value
R6 =0x0000ffff9bed85c0: <offset 0x00000000012725c0> in /usr/share/elasticsearch/jdk/lib/server/libjvm.so at 0x0000ffff9ac66000
R7 =0x000000000000000d is an unknown value
R8 =0x0 is null
R9 =0x000000000000000c is an unknown value
R10=0x000000000000000d is an unknown value
R11=0x00000004004b4ab0 is pointing into metadata
R12={method} {0x000000040017cc68} 'registerNatives' '()V' in 'java/lang/System'
R13=0x000000040073aa50 is pointing into metadata
R14=0x0 is null
R15=0x0 is null
R16=0x0000ffff9bec5858: <offset 0x000000000125f858> in /usr/share/elasticsearch/jdk/lib/server/libjvm.so at 0x0000ffff9ac66000
R17=0x0000ffff9c008b00: memcmp+0x0000000000000000 in /lib/aarch64-linux-gnu/libc.so.6 at 0x0000ffff9bf8a000
R18={method} {0x000000040017cb50} 'arraycopy' '(Ljava/lang/Object;ILjava/lang/Object;II)V' in 'java/lang/System'
R19=0x0000ffff83d3dc30 is at code_begin+48 in an Interpreter codelet
result handlers for native calls  [0x0000ffff83d3dc00, 0x0000ffff83d3dc48]  72 bytes
R20=0x0000ffff9aba10b0 is pointing into the stack for thread: 0x0000ffff9402a290
R21=0x0000ffff9bf74aa0: <offset 0x000000000130eaa0> in /usr/share/elasticsearch/jdk/lib/server/libjvm.so at 0x0000ffff9ac66000
R22=0x0 is null
R23=0x0000ffff83d3eec0 is at code_begin+0 in an Interpreter codelet
method entry point (kind = zerolocals)  [0x0000ffff83d3eec0, 0x0000ffff83d3f260]  928 bytes
R24=0x0000ffff9aba1138 is pointing into the stack for thread: 0x0000ffff9402a290
R25=0x0000ffff9aba1330 is pointing into the stack for thread: 0x0000ffff9402a290
R26=0x0000000400165758 is pointing into metadata
R27=0x0 is null
R28=0x0000ffff9402a290 is a thread
R29=0x0000ffff9aba1120 is pointing into the stack for thread: 0x0000ffff9402a290
R30=0x0000ffff83d4000c is at code_begin+908 in an Interpreter codelet
native method entry point (kind = native)  [0x0000ffff83d3fc80, 0x0000ffff83d403e0]  1888 bytes
R31=0x0000ffff9aba10b0 is pointing into the stack for thread: 0x0000ffff9402a290

Top of Stack: (sp=0x0000ffff9aba10b0)
0x0000ffff9aba10b0:   0000000000000000 0000000000000000
0x0000ffff9aba10c0:   200000ffff9aba11 9562cc5b923f0400
0x0000ffff9aba10d0:   fffffffffffffff6 0000000000000000
0x0000ffff9aba10e0:   0000000000000003 0000000400165758
0x0000ffff9aba10f0:   00000000fd5bf740 fffffffffffffff4
0x0000ffff9aba1100:   0000000000000000 000000040017cc68
0x0000ffff9aba1110:   0000000000000000 0000ffff9aba1120
0x0000ffff9aba1120:   0000ffff9aba1190 0000ffff83d3c110
0x0000ffff9aba1130:   0000000000000000 00000000fd5bf740
0x0000ffff9aba1140:   fffffffffffffff6 000000040073b0d8
0x0000ffff9aba1150:   0000000000000001 0000000400165758
0x0000ffff9aba1160:   00000000fd5bf740 fffffffffffffff2
0x0000ffff9aba1170:   0000000000000000 000000040017c860
0x0000ffff9aba1180:   fffffffffffffff6 0000ffff9aba11a0
0x0000ffff9aba1190:   0000ffff9aba1280 0000ffff83d37118
0x0000ffff9aba11a0:   0000000000000000 0000ffff9be059d8
0x0000ffff9aba11b0:   0000000000000000 0000000000000000
0x0000ffff9aba11c0:   0000000000000000 0000000000000000
0x0000ffff9aba11d0:   0000000000000000 0000000000000000
0x0000ffff9aba11e0:   0000000000000000 0000000000000000
0x0000ffff9aba11f0:   0000ffff9aba13f8 000000000000000a
0x0000ffff9aba1200:   0000ffff9aba12f8 0000ffff9aba1330
0x0000ffff9aba1210:   000000000000000e 0000ffff83d3eec0
0x0000ffff9aba1220:   0000ffff9aba13e8 0000ffff9aba13f0
0x0000ffff9aba1230:   0000ffff9aba13d0 0000ffff9402a290
0x0000ffff9aba1240:   0000ffff9aba1330 0000ffff9aba13e8
0x0000ffff9aba1250:   000000000000000a 000000040017c860
0x0000ffff9aba1260:   0000ffff83d3eec0 0000ffff9aba13f8
0x0000ffff9aba1270:   0000ffff00000000 0000ffff9402a290
0x0000ffff9aba1280:   0000ffff9aba1290 0000ffff9b43d368
0x0000ffff9aba1290:   0000ffff9aba1370 0000ffff9b4178f4
0x0000ffff9aba12a0:   0000ffff9402a290 0000ffff9aba13d0 

Instructions: (pc=0x0000ffff83d40028)
0x0000ffff83d3ff28:   b8 6f 38 8b 91 31 40 f9 20 02 3f d6 ac 83 5e f8
0x0000ffff83d3ff38:   f3 03 00 aa 91 29 40 b9 11 01 18 36 91 05 40 f9
0x0000ffff83d3ff48:   31 06 40 f9 31 0e 40 f9 31 3a 40 f9 31 02 40 f9
0x0000ffff83d3ff58:   b1 0f 00 f9 a1 63 00 91 8a 2d 40 f9 09 56 97 d2
0x0000ffff83d3ff68:   09 71 b3 f2 e9 ff df f2 29 01 40 f9 5f 01 09 eb
0x0000ffff83d3ff78:   81 03 00 54 e1 03 0c aa b6 83 1b f8 e0 03 1c aa
0x0000ffff83d3ff88:   28 01 00 10 88 d7 01 f9 94 d3 01 f9 9d db 01 f9
0x0000ffff83d3ff98:   e8 33 bf a9 88 52 87 d2 68 68 b3 f2 e8 ff df f2
0x0000ffff83d3ffa8:   00 01 3f d6 e8 33 c1 a8 fe ff ff 10 9f d3 01 f9
0x0000ffff83d3ffb8:   9f db 01 f9 9f d7 01 f9 88 07 40 f9 a8 00 00 b4
0x0000ffff83d3ffc8:   08 08 8e d2 68 7a b0 f2 e8 ff df f2 00 01 1f d6
0x0000ffff83d3ffd8:   b6 e3 7b a9 b8 6f 38 8b ac 83 5e f8 8a 2d 40 f9
0x0000ffff83d3ffe8:   80 03 0f 91 08 01 00 10 88 d7 01 f9 94 d3 01 f9
0x0000ffff83d3fff8:   9d db 01 f9 e8 03 7e b2 89 13 11 91 28 fd 9f 88
0x0000ffff83d40008:   40 01 3f d6 ac 83 5e f8 88 82 00 d1 1f ed 7c 92
0x0000ffff83d40018:   80 0e 1f fc 9f 8e 1f f8 80 8e 1f f8 e8 03 1f 2a
0x0000ffff83d40028:   e8 e3 30 04 1f 05 00 31 80 00 00 54 c1 d5 bb d4
0x0000ffff83d40038:   30 2a b3 9b ff ff 00 00 a8 00 80 d2 89 13 11 91
0x0000ffff83d40048:   28 fd 9f 88 bf 3b 03 d5 88 23 11 91 08 fd df c8
0x0000ffff83d40058:   bf 03 08 eb 68 00 00 54 89 43 44 b9 69 01 00 b4
0x0000ffff83d40068:   e0 03 1c aa 89 d2 8d d2 a9 68 b3 f2 e9 ff df f2
0x0000ffff83d40078:   20 01 3f d6 ac 83 5e f8 1b 7a 94 d2 9b 7d b3 f2
0x0000ffff83d40088:   fb ff df f2 7b 03 40 f9 e8 03 7d b2 89 13 11 91
0x0000ffff83d40098:   28 fd 9f 88 9f d3 01 f9 9f db 01 f9 9f d7 01 f9
0x0000ffff83d400a8:   91 1b 42 f9 3f 82 00 f9 71 dc fe 10 3f 02 13 eb
0x0000ffff83d400b8:   c1 01 00 54 80 06 41 f8 20 01 00 b4 1f 04 40 f2
0x0000ffff83d400c8:   61 00 00 54 00 00 40 f9 05 00 00 14 60 00 00 37
0x0000ffff83d400d8:   00 e0 5f f8 02 00 00 14 00 f0 5f f8 a0 0f 00 f9
0x0000ffff83d400e8:   9f 8e 1f f8 80 8e 1f f8 88 23 13 91 08 01 40 b9
0x0000ffff83d400f8:   1f 09 00 f1 e1 04 00 54 e0 07 b7 a9 e2 0f 01 a9
0x0000ffff83d40108:   e4 17 02 a9 e6 1f 03 a9 ea 2f 04 a9 ec 37 05 a9
0x0000ffff83d40118:   ee 3f 06 a9 f0 47 07 a9 f2 7f 08 a9 ff 83 00 d1 


Stack slot to memory mapping:

stack at sp + 0 slots: 0x0 is null
stack at sp + 1 slots: 0x0 is null
stack at sp + 2 slots: 0x200000ffff9aba11 is an unknown value
stack at sp + 3 slots: 0x9562cc5b923f0400 is an unknown value
stack at sp + 4 slots: 0xfffffffffffffff6 is an unknown value
stack at sp + 5 slots: 0x0 is null
stack at sp + 6 slots: 0x0000000000000003 is an unknown value
stack at sp + 7 slots: 0x0000000400165758 is pointing into metadata

Lock stack of current Java thread (top to bottom):

native method entry point (kind = native)  [0x0000ffff83d3fc80, 0x0000ffff83d403e0]  1888 bytes
[MachCode]
  0x0000ffff83d3fc80: 8205 40f9 | 425c 4079 | 986e 228b | 1823 00d1 | 9fee 7c92 | f4c3 01d1 | f603 1faa | 2801 8092 
  0x0000ffff83d3fca0: e87f b9a9 | ff7f 06a9 | 8809 40f9 | 4800 00b4 | 0801 0591 | e833 03a9 | fd7b 05a9 | fd43 0191 
  0x0000ffff83d3fcc0: 9a05 40f9 | 5a07 40f9 | 5a0b 40f9 | 0803 1dcb | 08fd 43d3 | e86b 01a9 | ff4f 04a9 | 8a05 40f9 
  0x0000ffff83d3fce0: 4a05 40f9 | 4a0d 40f9 | 4a39 40f9 | 4a01 40f9 | e843 00d1 | 0901 1dcb | 29fd 4393 | ea27 02a9 
  0x0000ffff83d3fd00: 1f01 0091 | e903 40b2 | 89c7 1139 | 8009 40f9 | 0001 00b4 | 08fc 40b9 | 0809 0091 | 08fc 00b9 
  0x0000ffff83d3fd20: 0910 41b9 | 0801 09ea | 8032 0054 | 2400 0014 | 890d 40f9 | 8903 00b5 | e103 0caa | b683 1bf8 
  0x0000ffff83d3fd40: e003 1caa | 2801 0010 | 88d7 01f9 | 94d3 01f9 | 9ddb 01f9 | e833 bfa9 | 083e 82d2 | 6868 b3f2 
  0x0000ffff83d3fd60: e8ff dff2 | 0001 3fd6 | e833 c1a8 | feff ff10 | 9fd3 01f9 | 9fdb 01f9 | 9fd7 01f9 | 8807 40f9 
  0x0000ffff83d3fd80: a800 00b4 | 0808 8ed2 | 687a b0f2 | e8ff dff2 | 0001 1fd6 | b6e3 7ba9 | b86f 388b | 890d 40f9 
  0x0000ffff83d3fda0: e900 00b4 | 2809 40b9 | 0809 0091 | 2809 00b9 | 211d 40b9 | 0801 01ea | 002e 0054 | 8877 42f9 
  0x0000ffff83d3fdc0: ff63 28eb | c805 0054 | e907 40d1 | 3f01 00f9 | e90b 40d1 | 3f01 00f9 | e90f 40d1 | 3f01 00f9 
  0x0000ffff83d3fde0: e913 40d1 | 3f01 00f9 | e917 40d1 | 3f01 00f9 | e91b 40d1 | 3f01 00f9 | e91f 40d1 | 3f01 00f9 
  0x0000ffff83d3fe00: e923 40d1 | 3f01 00f9 | e927 40d1 | 3f01 00f9 | e92b 40d1 | 3f01 00f9 | e92f 40d1 | 3f01 00f9 
  0x0000ffff83d3fe20: e933 40d1 | 3f01 00f9 | e937 40d1 | 3f01 00f9 | e93b 40d1 | 3f01 00f9 | e93f 40d1 | 3f01 00f9 
  0x0000ffff83d3fe40: e943 40d1 | 3f01 00f9 | e947 40d1 | 3f01 00f9 | e94b 40d1 | 3f01 00f9 | e94f 40d1 | 3f01 00f9 
  0x0000ffff83d3fe60: e953 40d1 | 3f01 00f9 | 8873 42f9 | ff63 28eb | 6900 0054 | e803 0091 | 8877 02f9 | 9fc7 1139 
  0x0000ffff83d3fe80: 480e 0cf0 | 08b5 7239 | 2801 0034 | a183 5ef8 | e003 1caa | e833 bfa9 | 081e 9ad2 | 0871 b3f2 
  0x0000ffff83d3fea0: e8ff dff2 | 0001 3fd6 | e833 c1a8 | 9105 40f9 | 315e 4079 | 886e 31cb | 1fed 7c92 | f403 08aa 
  0x0000ffff83d3fec0: 9131 40f9 | 7103 00b5 | e103 0caa | b683 1bf8 | e003 1caa | 2801 0010 | 88d7 01f9 | 94d3 01f9 
  0x0000ffff83d3fee0: 9ddb 01f9 | e833 bfa9 | 8852 87d2 | 6868 b3f2 | e8ff dff2 | 0001 3fd6 | e833 c1a8 | feff ff10 
  0x0000ffff83d3ff00: 9fd3 01f9 | 9fdb 01f9 | 9fd7 01f9 | 8807 40f9 | a800 00b4 | 0808 8ed2 | 687a b0f2 | e8ff dff2 
  0x0000ffff83d3ff20: 0001 1fd6 | b6e3 7ba9 | b86f 388b | 9131 40f9 | 2002 3fd6 | ac83 5ef8 | f303 00aa | 9129 40b9 
  0x0000ffff83d3ff40: 1101 1836 | 9105 40f9 | 3106 40f9 | 310e 40f9 | 313a 40f9 | 3102 40f9 | b10f 00f9 | a163 0091 
  0x0000ffff83d3ff60: 8a2d 40f9 | 0956 97d2 | 0971 b3f2 | e9ff dff2 | 2901 40f9 | 5f01 09eb | 8103 0054 | e103 0caa 
  0x0000ffff83d3ff80: b683 1bf8 | e003 1caa | 2801 0010 | 88d7 01f9 | 94d3 01f9 | 9ddb 01f9 | e833 bfa9 | 8852 87d2 
  0x0000ffff83d3ffa0: 6868 b3f2 | e8ff dff2 | 0001 3fd6 | e833 c1a8 | feff ff10 | 9fd3 01f9 | 9fdb 01f9 | 9fd7 01f9 
  0x0000ffff83d3ffc0: 8807 40f9 | a800 00b4 | 0808 8ed2 | 687a b0f2 | e8ff dff2 | 0001 1fd6 | b6e3 7ba9 | b86f 388b 
  0x0000ffff83d3ffe0: ac83 5ef8 | 8a2d 40f9 | 8003 0f91 | 0801 0010 | 88d7 01f9 | 94d3 01f9 | 9ddb 01f9 | e803 7eb2 
  0x0000ffff83d40000: 8913 1191 | 28fd 9f88 | 4001 3fd6 | ac83 5ef8 | 8882 00d1 | 1fed 7c92 | 800e 1ffc | 9f8e 1ff8 
  0x0000ffff83d40020: 808e 1ff8 | e803 1f2a | e8e3 3004 | 1f05 0031 | 8000 0054 | c1d5 bbd4 | 302a b39b | ffff 0000 
  0x0000ffff83d40040: a800 80d2 | 8913 1191 | 28fd 9f88 | bf3b 03d5 | 8823 1191 | 08fd dfc8 | bf03 08eb | 6800 0054 
  0x0000ffff83d40060: 8943 44b9 | 6901 00b4 | e003 1caa | 89d2 8dd2 | a968 b3f2 | e9ff dff2 | 2001 3fd6 | ac83 5ef8 
  0x0000ffff83d40080: 1b7a 94d2 | 9b7d b3f2 | fbff dff2 | 7b03 40f9 | e803 7db2 | 8913 1191 | 28fd 9f88 | 9fd3 01f9 
  0x0000ffff83d400a0: 9fdb 01f9 | 9fd7 01f9 | 911b 42f9 | 3f82 00f9 | 71dc fe10 | 3f02 13eb | c101 0054 | 8006 41f8 
  0x0000ffff83d400c0: 2001 00b4 | 1f04 40f2 | 6100 0054 | 0000 40f9 | 0500 0014 | 6000 0037 | 00e0 5ff8 | 0200 0014 
  0x0000ffff83d400e0: 00f0 5ff8 | a00f 00f9 | 9f8e 1ff8 | 808e 1ff8 | 8823 1391 | 0801 40b9 | 1f09 00f1 | e104 0054 
  0x0000ffff83d40100: e007 b7a9 | e20f 01a9 | e417 02a9 | e61f 03a9 | ea2f 04a9 | ec37 05a9 | ee3f 06a9 | f047 07a9 
  0x0000ffff83d40120: f27f 08a9 | ff83 00d1 | e8eb 7bb2 | fc2f 880c | f82f 880c | f42f 880c | f02f 880c | e42f 880c 
  0x0000ffff83d40140: e02f 000c | e003 1caa | 89f8 9ad2 | 0971 b3f2 | e9ff dff2 | 2001 3fd6 | e02f df0c | e42f df0c 
  0x0000ffff83d40160: f02f df0c | f42f df0c | f82f df0c | fc2f df0c | e7e3 1825 | e20f 41a9 | e417 42a9 | e61f 43a9 
  0x0000ffff83d40180: ea2f 44a9 | ec37 45a9 | ee3f 46a9 | f047 47a9 | f27f 48a9 | e007 c9a8 | ac83 5ef8 | 9605 40f9 
  0x0000ffff83d401a0: d6e2 0091 | 8807 40f9 | 8803 00b4 | b683 1bf8 | e003 1caa | 2801 0010 | 88d7 01f9 | 94d3 01f9 
  0x0000ffff83d401c0: 9ddb 01f9 | e833 bfa9 | 082c 80d2 | 6868 b3f2 | e8ff dff2 | 0001 3fd6 | e833 c1a8 | feff ff10 
  0x0000ffff83d401e0: 9fd3 01f9 | 9fdb 01f9 | 9fd7 01f9 | 8807 40f9 | a800 00b4 | 0808 8ed2 | 687a b0f2 | e8ff dff2 
  0x0000ffff83d40200: 0001 1fd6 | b6e3 7ba9 | b86f 388b | c1d5 bbd4 | d8c8 ad9b | ffff 0000 | 9129 40b9 | 9108 2836 
  0x0000ffff83d40220: a183 01d1 | 3104 40f9 | 9103 00b5 | b683 1bf8 | e003 1caa | 2801 0010 | 88d7 01f9 | 94d3 01f9 
  0x0000ffff83d40240: 9ddb 01f9 | e833 bfa9 | 8842 81d2 | 6868 b3f2 | e8ff dff2 | 0001 3fd6 | e833 c1a8 | feff ff10 
  0x0000ffff83d40260: 9fd3 01f9 | 9fdb 01f9 | 9fd7 01f9 | 8807 40f9 | a800 00b4 | 0808 8ed2 | 687a b0f2 | e8ff dff2 
  0x0000ffff83d40280: 0001 1fd6 | b6e3 7ba9 | b86f 388b | c1d5 bbd4 | d8c8 ad9b | ffff 0000 | b683 1bf8 | 2304 40f9 
  0x0000ffff83d402a0: 3f04 00f9 | 82cb 46b9 | 4220 0051 | 846b 62f8 | 7f00 04eb | 2102 0054 | 82cb 06b9 | 4420 0051 
  0x0000ffff83d402c0: 846b 64f8 | 7f00 04eb | 6001 0054 | 6000 40f9 | c000 0837 | 0400 40b2 | e803 00aa | 64fc a8c8 
  0x0000ffff83d402e0: 1f01 00eb | 8000 0054 | 4220 0011 | 82cb 06b9 | 0200 0014 | 0a00 0014 | 2304 00f9 | e003 01aa 
  0x0000ffff83d40300: e833 bfa9 | 082e 81d2 | 6868 b3f2 | e8ff dff2 | 0001 3fd6 | e833 c1a8 | 0400 0014 | 88ab 42f9 
  0x0000ffff83d40320: 0805 00d1 | 88ab 02f9 | b683 5bf8 | 480e 0cd0 | 08b5 7239 | 2801 0034 | a183 5ef8 | e003 1caa 
  0x0000ffff83d40340: e833 bfa9 | 0822 9ad2 | 0871 b3f2 | e8ff dff2 | 0001 3fd6 | e833 c1a8 | 8006 41f8 | 8006 41fc 
  0x0000ffff83d40360: 6002 3fd6 | b483 5ff8 | bf03 0091 | fd7b c1a8 | 9f02 0091 | c003 5fd6 | 0100 80d2 | b683 1bf8 
  0x0000ffff83d40380: e003 1caa | 2801 0010 | 88d7 01f9 | 94d3 01f9 | 9ddb 01f9 | e833 bfa9 | 0832 88d2 | 6868 b3f2 
  0x0000ffff83d403a0: e8ff dff2 | 0001 3fd6 | e833 c1a8 | feff ff10 | 9fd3 01f9 | 9fdb 01f9 | 9fd7 01f9 | 8807 40f9 
  0x0000ffff83d403c0: a800 00b4 | 0808 8ed2 | 687a b0f2 | e8ff dff2 | 0001 1fd6 | b6e3 7ba9 | b86f 388b | 78fe ff17 
[/MachCode]

---------------  P R O C E S S  ---------------

Threads class SMR info:
_java_thread_list=0x0000ffff94056cc0, length=1, elements={
0x0000ffff9402a290
}

Java Threads: ( => current thread )
=>0x0000ffff9402a290 JavaThread "Unknown thread"             [_thread_in_native, id=18, stack(0x0000ffff9a9a5000,0x0000ffff9aba3000) (2040K)]
Total: 1

Other Threads:
  0x0000ffff94066e50 VMThread "VM Thread"                           [id=20, stack(0x0000ffff9a1d5000,0x0000ffff9a3d3000) (2040K)]
  0x0000ffff940561f0 WatcherThread "VM Periodic Task Thread"        [id=19, stack(0x0000ffff9a3d4000,0x0000ffff9a5d2000) (2040K)]
Total: 2

Threads with active compile tasks:
Total: 0

VM state: not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: None

Heap address: 0x00000000fc000000, size: 64 MB, Compressed Oops mode: 32-bit

CDS archive(s) mapped at: [0x0000000400000000-0x0000000400da0000-0x0000000400da0000), size 14286848, SharedBaseAddress: 0x0000000400000000, ArchiveRelocationMode: 1.
Compressed class space mapped at: 0x0000000401000000-0x0000000441000000, reserved size: 1073741824
Narrow klass base: 0x0000000400000000, Narrow klass shift: 0, Narrow klass range: 0x100000000

GC Precious Log:
 CardTable entry size: 512
 CPUs: 10 total, 10 available
 Memory: 7836M
 Large Page Support: Disabled
 NUMA Support: Disabled
 Compressed Oops: Enabled (32-bit)
 Heap Min Capacity: 4M
 Heap Initial Capacity: 4M
 Heap Max Capacity: 64M
 Pre-touch: Disabled

Heap:
 def new generation   total 1216K, used 21K [0x00000000fc000000, 0x00000000fc150000, 0x00000000fd550000)
  eden space 1088K,   2% used [0x00000000fc000000, 0x00000000fc005718, 0x00000000fc110000)
  from space 128K,   0% used [0x00000000fc110000, 0x00000000fc110000, 0x00000000fc130000)
  to   space 128K,   0% used [0x00000000fc130000, 0x00000000fc130000, 0x00000000fc150000)
 tenured generation   total 2752K, used 1145K [0x00000000fd550000, 0x00000000fd800000, 0x0000000100000000)
   the space 2752K,  41% used [0x00000000fd550000, 0x00000000fd66e6b0, 0x00000000fd800000)
 Metaspace       used 1K, committed 128K, reserved 1114112K
  class space    used 0K, committed 64K, reserved 1048576K

Card table byte_map: [0x0000ffff9a705000,0x0000ffff9a725000] _byte_map_base: 0x0000ffff99f25000

Polling page: 0x0000ffff9c1b5000

Metaspace:

Usage:
  Non-class:    304 bytes used.
      Class:   1008 bytes used.
       Both:      1.28 KB used.

Virtual space:
  Non-class space:       64.00 MB reserved,      64.00 KB ( <1%) committed,  1 nodes.
      Class space:        1.00 GB reserved,      64.00 KB ( <1%) committed,  1 nodes.
             Both:        1.06 GB reserved,     128.00 KB ( <1%) committed. 

Chunk freelists:
   Non-Class:  12.00 MB
       Class:  15.75 MB
        Both:  27.75 MB

MaxMetaspaceSize: unlimited
CompressedClassSpaceSize: 1.00 GB
Initial GC threshold: 21.00 MB
Current GC threshold: 17179869184.00 GB
CDS: on
 - commit_granule_bytes: 65536.
 - commit_granule_words: 8192.
 - virtual_space_node_default_size: 8388608.
 - enlarge_chunks_in_place: 1.
 - use_allocation_guard: 0.


Internal statistics:

num_allocs_failed_limit: 0.
num_arena_births: 2.
num_arena_deaths: 0.
num_vsnodes_births: 2.
num_vsnodes_deaths: 0.
num_space_committed: 2.
num_space_uncommitted: 0.
num_chunks_returned_to_freelist: 0.
num_chunks_taken_from_freelist: 3.
num_chunk_merges: 0.
num_chunk_splits: 2.
num_chunks_enlarged: 0.
num_inconsistent_stats: 0.

CodeHeap 'non-profiled nmethods': size=120032Kb used=0Kb max_used=0Kb free=120032Kb
 bounds [0x0000ffff842c8000, 0x0000ffff84538000, 0x0000ffff8b800000]
CodeHeap 'profiled nmethods': size=120028Kb used=0Kb max_used=0Kb free=120028Kb
 bounds [0x0000ffff7c800000, 0x0000ffff7ca70000, 0x0000ffff83d37000]
CodeHeap 'non-nmethods': size=5700Kb used=359Kb max_used=359Kb free=5340Kb
 bounds [0x0000ffff83d37000, 0x0000ffff83fa7000, 0x0000ffff842c8000]
CodeCache: size=245760Kb, used=359Kb, max_used=359Kb, free=245400Kb
 total_blobs=91, nmethods=0, adapters=67, full_count=0
Compilation: enabled, stopped_count=0, restarted_count=0

Compilation events (0 events):
No events

GC Heap History (0 events):
No events

Dll operation events (1 events):
Event: 0.001 Loaded shared library /usr/share/elasticsearch/jdk/lib/libjava.so

Deoptimization events (0 events):
No events

Classes loaded (0 events):
No events

Classes unloaded (0 events):
No events

Classes redefined (0 events):
No events

Internal exceptions (0 events):
No events

VM Operations (0 events):
No events

Memory protections (1 events):
Event: 0.001 Protecting memory [0x0000ffff9a9a5000,0x0000ffff9a9a9000] with protection modes 0

Nmethod flushes (0 events):
No events

Events (1 events):
Event: 0.005 Thread 0x0000ffff9402a290 Thread added: 0x0000ffff9402a290


Dynamic libraries:
fc000000-fc150000 rw-p 00000000 00:00 0 
fc150000-fd550000 ---p 00000000 00:00 0 
fd550000-fd800000 rw-p 00000000 00:00 0 
fd800000-100000000 ---p 00000000 00:00 0 
400000000-4004b0000 rw-p 00010000 00:42 33559230                         /usr/share/elasticsearch/jdk/lib/server/classes.jsa
4004b0000-400da0000 rw-p 004c0000 00:42 33559230                         /usr/share/elasticsearch/jdk/lib/server/classes.jsa
400da0000-401040000 ---p 00000000 00:00 0 
401040000-401050000 rw-p 00000000 00:00 0 
401050000-441000000 ---p 00000000 00:00 0 
aaaabee50000-aaaabee51000 r-xp 00000000 00:42 33558733                   /usr/share/elasticsearch/jdk/bin/java
aaaabee6f000-aaaabee70000 r--p 0000f000 00:42 33558733                   /usr/share/elasticsearch/jdk/bin/java
aaaabee70000-aaaabee71000 rw-p 00010000 00:42 33558733                   /usr/share/elasticsearch/jdk/bin/java
aaaae3e07000-aaaae3e56000 rw-p 00000000 00:00 0                          [heap]
ffff6c000000-ffff6c021000 rw-p 00000000 00:00 0 
ffff6c021000-ffff70000000 ---p 00000000 00:00 0 
ffff74000000-ffff74010000 rw-p 00000000 00:00 0 
ffff74010000-ffff78000000 ---p 00000000 00:00 0 
ffff78000000-ffff78021000 rw-p 00000000 00:00 0 
ffff78021000-ffff7c000000 ---p 00000000 00:00 0 
ffff7c800000-ffff7ca70000 rwxp 00000000 00:00 0 
ffff7ca70000-ffff83d37000 ---p 00000000 00:00 0 
ffff83d37000-ffff83fa7000 rwxp 00000000 00:00 0 
ffff83fa7000-ffff842c8000 ---p 00000000 00:00 0 
ffff842c8000-ffff84538000 rwxp 00000000 00:00 0 
ffff84538000-ffff8b800000 ---p 00000000 00:00 0 
ffff8b841000-ffff94000000 r--s 00000000 00:42 33559221                   /usr/share/elasticsearch/jdk/lib/modules
ffff94000000-ffff94085000 rw-p 00000000 00:00 0 
ffff94085000-ffff98000000 ---p 00000000 00:00 0 
ffff99d49000-ffff9a1d4000 rw-p 00000000 00:00 0 
ffff9a1d4000-ffff9a1d5000 ---p 00000000 00:00 0 
ffff9a1d5000-ffff9a3d3000 rw-p 00000000 00:00 0 
ffff9a3d3000-ffff9a3d4000 ---p 00000000 00:00 0 
ffff9a3d4000-ffff9a5d2000 rw-p 00000000 00:00 0 
ffff9a5d2000-ffff9a6f1000 rw-p 00000000 00:00 0 
ffff9a6f1000-ffff9a705000 ---p 00000000 00:00 0 
ffff9a705000-ffff9a706000 rw-p 00000000 00:00 0 
ffff9a706000-ffff9a70f000 ---p 00000000 00:00 0 
ffff9a70f000-ffff9a711000 rw-p 00000000 00:00 0 
ffff9a711000-ffff9a725000 ---p 00000000 00:00 0 
ffff9a725000-ffff9a72a000 rw-p 00000000 00:00 0 
ffff9a72a000-ffff9a810000 ---p 00000000 00:00 0 
ffff9a810000-ffff9a815000 rw-p 00000000 00:00 0 
ffff9a815000-ffff9a81c000 ---p 00000000 00:00 0 
ffff9a81c000-ffff9a821000 rw-p 00000000 00:00 0 
ffff9a821000-ffff9a907000 ---p 00000000 00:00 0 
ffff9a907000-ffff9a928000 r-xp 00000000 00:42 33559198                   /usr/share/elasticsearch/jdk/lib/libjava.so
ffff9a928000-ffff9a946000 ---p 00021000 00:42 33559198                   /usr/share/elasticsearch/jdk/lib/libjava.so
ffff9a946000-ffff9a947000 r--p 0002f000 00:42 33559198                   /usr/share/elasticsearch/jdk/lib/libjava.so
ffff9a947000-ffff9a948000 rw-p 00030000 00:42 33559198                   /usr/share/elasticsearch/jdk/lib/libjava.so
ffff9a948000-ffff9a949000 rw-p 00000000 00:00 0 
ffff9a949000-ffff9a951000 rw-s 00000000 00:42 33560302                   /tmp/hsperfdata_root/9
ffff9a951000-ffff9a95c000 r-xp 00000000 00:42 33555324                   /usr/lib/aarch64-linux-gnu/libnss_files-2.31.so
ffff9a95c000-ffff9a96c000 ---p 0000b000 00:42 33555324                   /usr/lib/aarch64-linux-gnu/libnss_files-2.31.so
ffff9a96c000-ffff9a96d000 r--p 0000b000 00:42 33555324                   /usr/lib/aarch64-linux-gnu/libnss_files-2.31.so
ffff9a96d000-ffff9a96e000 rw-p 0000c000 00:42 33555324                   /usr/lib/aarch64-linux-gnu/libnss_files-2.31.so
ffff9a96e000-ffff9a974000 rw-p 00000000 00:00 0 
ffff9a974000-ffff9a991000 r-xp 00000000 00:42 33559202                   /usr/share/elasticsearch/jdk/lib/libjimage.so
ffff9a991000-ffff9a9a2000 ---p 0001d000 00:42 33559202                   /usr/share/elasticsearch/jdk/lib/libjimage.so
ffff9a9a2000-ffff9a9a4000 r--p 0001e000 00:42 33559202                   /usr/share/elasticsearch/jdk/lib/libjimage.so
ffff9a9a4000-ffff9a9a5000 rw-p 00020000 00:42 33559202                   /usr/share/elasticsearch/jdk/lib/libjimage.so
ffff9a9a5000-ffff9a9a9000 ---p 00000000 00:00 0 
ffff9a9a9000-ffff9aba3000 rw-p 00000000 00:00 0 
ffff9aba3000-ffff9ac3c000 r-xp 00000000 00:42 33555303                   /usr/lib/aarch64-linux-gnu/libm-2.31.so
ffff9ac3c000-ffff9ac4c000 ---p 00099000 00:42 33555303                   /usr/lib/aarch64-linux-gnu/libm-2.31.so
ffff9ac4c000-ffff9ac4d000 r--p 00099000 00:42 33555303                   /usr/lib/aarch64-linux-gnu/libm-2.31.so
ffff9ac4d000-ffff9ac4e000 rw-p 0009a000 00:42 33555303                   /usr/lib/aarch64-linux-gnu/libm-2.31.so
ffff9ac4e000-ffff9ac55000 r-xp 00000000 00:42 33555357                   /usr/lib/aarch64-linux-gnu/librt-2.31.so
ffff9ac55000-ffff9ac64000 ---p 00007000 00:42 33555357                   /usr/lib/aarch64-linux-gnu/librt-2.31.so
ffff9ac64000-ffff9ac65000 r--p 00006000 00:42 33555357                   /usr/lib/aarch64-linux-gnu/librt-2.31.so
ffff9ac65000-ffff9ac66000 rw-p 00007000 00:42 33555357                   /usr/lib/aarch64-linux-gnu/librt-2.31.so
ffff9ac66000-ffff9bdd6000 r-xp 00000000 00:42 33559233                   /usr/share/elasticsearch/jdk/lib/server/libjvm.so
ffff9bdd6000-ffff9bdf3000 ---p 01170000 00:42 33559233                   /usr/share/elasticsearch/jdk/lib/server/libjvm.so
ffff9bdf3000-ffff9bec6000 r--p 0117d000 00:42 33559233                   /usr/share/elasticsearch/jdk/lib/server/libjvm.so
ffff9bec6000-ffff9bef5000 rw-p 01250000 00:42 33559233                   /usr/share/elasticsearch/jdk/lib/server/libjvm.so
ffff9bef5000-ffff9bf8a000 rw-p 00000000 00:00 0 
ffff9bf8a000-ffff9c0e5000 r-xp 00000000 00:42 33555261                   /usr/lib/aarch64-linux-gnu/libc-2.31.so
ffff9c0e5000-ffff9c0f4000 ---p 0015b000 00:42 33555261                   /usr/lib/aarch64-linux-gnu/libc-2.31.so
ffff9c0f4000-ffff9c0f8000 r--p 0015a000 00:42 33555261                   /usr/lib/aarch64-linux-gnu/libc-2.31.so
ffff9c0f8000-ffff9c0fa000 rw-p 0015e000 00:42 33555261                   /usr/lib/aarch64-linux-gnu/libc-2.31.so
ffff9c0fa000-ffff9c0fd000 rw-p 00000000 00:00 0 
ffff9c0fd000-ffff9c119000 r-xp 00000000 00:42 33555353                   /usr/lib/aarch64-linux-gnu/libpthread-2.31.so
ffff9c119000-ffff9c128000 ---p 0001c000 00:42 33555353                   /usr/lib/aarch64-linux-gnu/libpthread-2.31.so
ffff9c128000-ffff9c129000 r--p 0001b000 00:42 33555353                   /usr/lib/aarch64-linux-gnu/libpthread-2.31.so
ffff9c129000-ffff9c12a000 rw-p 0001c000 00:42 33555353                   /usr/lib/aarch64-linux-gnu/libpthread-2.31.so
ffff9c12a000-ffff9c12e000 rw-p 00000000 00:00 0 
ffff9c12e000-ffff9c131000 r-xp 00000000 00:42 33555272                   /usr/lib/aarch64-linux-gnu/libdl-2.31.so
ffff9c131000-ffff9c140000 ---p 00003000 00:42 33555272                   /usr/lib/aarch64-linux-gnu/libdl-2.31.so
ffff9c140000-ffff9c141000 r--p 00002000 00:42 33555272                   /usr/lib/aarch64-linux-gnu/libdl-2.31.so
ffff9c141000-ffff9c142000 rw-p 00003000 00:42 33555272                   /usr/lib/aarch64-linux-gnu/libdl-2.31.so
ffff9c142000-ffff9c15b000 r-xp 00000000 00:42 33555389                   /usr/lib/aarch64-linux-gnu/libz.so.1.2.11
ffff9c15b000-ffff9c16a000 ---p 00019000 00:42 33555389                   /usr/lib/aarch64-linux-gnu/libz.so.1.2.11
ffff9c16a000-ffff9c16b000 r--p 00018000 00:42 33555389                   /usr/lib/aarch64-linux-gnu/libz.so.1.2.11
ffff9c16b000-ffff9c16c000 rw-p 00019000 00:42 33555389                   /usr/lib/aarch64-linux-gnu/libz.so.1.2.11
ffff9c16c000-ffff9c17b000 r-xp 00000000 00:42 33559203                   /usr/share/elasticsearch/jdk/lib/libjli.so
ffff9c17b000-ffff9c18b000 ---p 0000f000 00:42 33559203                   /usr/share/elasticsearch/jdk/lib/libjli.so
ffff9c18b000-ffff9c18c000 r--p 0000f000 00:42 33559203                   /usr/share/elasticsearch/jdk/lib/libjli.so
ffff9c18c000-ffff9c18d000 rw-p 00010000 00:42 33559203                   /usr/share/elasticsearch/jdk/lib/libjli.so
ffff9c18d000-ffff9c1ae000 r-xp 00000000 00:42 33555239                   /usr/lib/aarch64-linux-gnu/ld-2.31.so
ffff9c1b3000-ffff9c1b5000 rw-p 00000000 00:00 0 
ffff9c1b5000-ffff9c1b6000 ---p 00000000 00:00 0 
ffff9c1b6000-ffff9c1b7000 r--p 00000000 00:00 0 
ffff9c1b7000-ffff9c1b8000 ---p 00000000 00:00 0 
ffff9c1b8000-ffff9c1ba000 rw-p 00000000 00:00 0 
ffff9c1ba000-ffff9c1bc000 r--p 00000000 00:00 0                          [vvar]
ffff9c1bc000-ffff9c1be000 r-xp 00000000 00:00 0                          [vdso]
ffff9c1be000-ffff9c1bf000 r--p 00021000 00:42 33555239                   /usr/lib/aarch64-linux-gnu/ld-2.31.so
ffff9c1bf000-ffff9c1c1000 rw-p 00022000 00:42 33555239                   /usr/lib/aarch64-linux-gnu/ld-2.31.so
ffffcd5fa000-ffffcd61b000 rw-p 00000000 00:00 0                          [stack]
Total number of mappings: 108


VM Arguments:
jvm_args: -Xms4m -Xmx64m -XX:+UseSerialGC -Dcli.name= -Dcli.script=bin/elasticsearch-certutil -Dcli.libs=modules/x-pack-core,modules/x-pack-security,lib/tools/security-cli -Des.path.home=/usr/share/elasticsearch -Des.path.conf=/usr/share/elasticsearch/config -Des.distribution.type=docker 
java_command: org.elasticsearch.launcher.CliToolLauncher ca --silent --pem -out config/certs/ca.zip
java_class_path (initial): /usr/share/elasticsearch/lib/lucene-grouping-9.12.0.jar:/usr/share/elasticsearch/lib/joni-2.1.29.jar:/usr/share/elasticsearch/lib/hppc-0.8.1.jar:/usr/share/elasticsearch/lib/elasticsearch-geo-8.17.1.jar:/usr/share/elasticsearch/lib/elasticsearch-plugin-api-8.17.1.jar:/usr/share/elasticsearch/lib/elasticsearch-entitlement-8.17.1.jar:/usr/share/elasticsearch/lib/elasticsearch-logging-8.17.1.jar:/usr/share/elasticsearch/lib/lucene-sandbox-9.12.0.jar:/usr/share/elasticsearch/lib/elasticsearch-plugin-analysis-api-8.17.1.jar:/usr/share/elasticsearch/lib/elasticsearch-lz4-8.17.1.jar:/usr/share/elasticsearch/lib/log4j2-ecs-layout-1.2.0.jar:/usr/share/elasticsearch/lib/lucene-facet-9.12.0.jar:/usr/share/elasticsearch/lib/elasticsearch-grok-8.17.1.jar:/usr/share/elasticsearch/lib/lucene-queries-9.12.0.jar:/usr/share/elasticsearch/lib/lucene-backward-codecs-9.12.0.jar:/usr/share/elasticsearch/lib/elasticsearch-native-8.17.1.jar:/usr/share/elasticsearch/lib/jopt-simple-5.0.2.jar:/usr/share/elasticsearch/lib/lucene-join-9.12.0.jar:/usr/share/elasticsearch/lib/lucene-misc-9.12.0.jar:/usr/share/elasticsearch/lib/lz4-java-1.8.0.jar:/usr/share/elasticsearch/lib/elasticsearch-8.17.1.jar:/usr/share/elasticsearch/lib/lucene-suggest-9.12.0.jar:/usr/share/elasticsearch/lib/elasticsearch-x-content-8.17.1.jar:/usr/share/elasticsearch/lib/lucene-core-9.12.0.jar:/usr/share/elasticsearch/lib/elasticsearch-tdigest-8.17.1.jar:/usr/share/elasticsearch/lib/jcodings-1.0.44.jar:/usr/share/elasticsearch/lib/log4j-api-2.19.0.jar:/usr/share/elasticsearch/lib/ecs-logging-core-1.2.0.jar:/usr/share/elasticsearch/lib/elasticsearch-cli-8.17.1.jar:/usr/share/elasticsearch/lib/elasticsearch-secure-sm-8.17.1.jar:/usr/share/elasticsearch/lib/elasticsearch-core-8.17.1.jar:/usr/share/elasticsearch/lib/lucene-queryparser-9.12.0.jar:/usr/share/elasticsearch/lib/elasticsearch-log4j-8.17.1.jar:/usr/share/elasticsearch/lib/lucene-analysis-common-9.12.0.jar:/usr/share/elasticsearch/lib/lucene-memory-9.12.0.jar:/usr/share/elasticsearch/lib/HdrHistogram-2.1.9.jar:/usr/share/elasticsearch/lib/elasticsearch-simdvec-8.17.1.jar:/usr/share/elasticsearch/lib/lucene-highlighter-9.12.0.jar:/usr/share/elasticsearch/lib/cli-launcher/cli-launcher-8.17.1.jar
Launcher Type: SUN_STANDARD

[Global flags]
     intx CICompilerCount                          = 4                                         {product} {ergonomic}
   size_t InitialHeapSize                          = 4194304                                   {product} {command line}
   size_t MaxHeapSize                              = 67108864                                  {product} {command line}
   size_t MaxNewSize                               = 22347776                                  {product} {ergonomic}
   size_t MinHeapDeltaBytes                        = 196608                                    {product} {ergonomic}
   size_t MinHeapSize                              = 4194304                                   {product} {command line}
   size_t NewSize                                  = 1376256                                   {product} {ergonomic}
    uintx NonNMethodCodeHeapSize                   = 5836800                                {pd product} {ergonomic}
    uintx NonProfiledCodeHeapSize                  = 122912768                              {pd product} {ergonomic}
   size_t OldSize                                  = 2818048                                   {product} {ergonomic}
    uintx ProfiledCodeHeapSize                     = 122908672                              {pd product} {ergonomic}
    uintx ReservedCodeCacheSize                    = 251658240                              {pd product} {ergonomic}
     bool SegmentedCodeCache                       = true                                      {product} {ergonomic}
   size_t SoftMaxHeapSize                          = 67108864                               {manageable} {ergonomic}
     bool UseCompressedOops                        = true                           {product lp64_product} {ergonomic}
     bool UseSerialGC                              = true                                      {product} {command line}

Logging:
Log output configuration:
 #0: stdout all=warning uptime,level,tags foldmultilines=false
 #1: stderr all=off uptime,level,tags foldmultilines=false

Environment Variables:
PATH=/usr/share/elasticsearch/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
SHELL=/bin/bash

Active Locale:
LC_ALL=C
LC_COLLATE=C
LC_CTYPE=C
LC_MESSAGES=C
LC_MONETARY=C
LC_NUMERIC=C
LC_TIME=C

Signal Handlers:
   SIGSEGV: crash_handler in libjvm.so, mask=11100100010111111101111111111110, flags=SA_RESTART|SA_SIGINFO, unblocked
    SIGBUS: crash_handler in libjvm.so, mask=11100100010111111101111111111110, flags=SA_RESTART|SA_SIGINFO, unblocked
    SIGFPE: crash_handler in libjvm.so, mask=11100100010111111101111111111110, flags=SA_RESTART|SA_SIGINFO, unblocked
   SIGPIPE: javaSignalHandler in libjvm.so, mask=11100100010111111101111111111110, flags=SA_RESTART|SA_SIGINFO, blocked
   SIGXFSZ: javaSignalHandler in libjvm.so, mask=11100100010111111101111111111110, flags=SA_RESTART|SA_SIGINFO, blocked
    SIGILL: crash_handler in libjvm.so, mask=11100100010111111101111111111110, flags=SA_RESTART|SA_SIGINFO, unblocked
   SIGUSR2: SR_handler in libjvm.so, mask=00000000000000000000000000000000, flags=SA_RESTART|SA_SIGINFO, blocked
    SIGHUP: SIG_DFL, mask=00000000000000000000000000000000, flags=none, blocked
    SIGINT: SIG_DFL, mask=00000000000000000000000000000000, flags=none, blocked
   SIGTERM: SIG_DFL, mask=00000000000000000000000000000000, flags=none, blocked
   SIGQUIT: UserHandler in libjvm.so, mask=11100100010111111101111111111110, flags=SA_RESTART|SA_SIGINFO, blocked
   SIGTRAP: crash_handler in libjvm.so, mask=11100100010111111101111111111110, flags=SA_RESTART|SA_SIGINFO, unblocked


Periodic native trim disabled

---------------  S Y S T E M  ---------------

OS:
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.6 LTS"
uname: Linux 6.12.5-linuxkit #1 SMP Tue Jan 21 10:23:32 UTC 2025 aarch64
OS uptime: 0 days 0:21 hours
libc: glibc 2.31 NPTL 2.31 
rlimit (soft/hard): STACK 8192k/infinity , CORE 0k/infinity , NPROC infinity/infinity , NOFILE 1048576/1048576 , AS infinity/infinity , CPU infinity/infinity , DATA infinity/infinity , FSIZE infinity/infinity , MEMLOCK infinity/infinity
load average: 0.16 0.28 0.22

/proc/meminfo:
MemTotal:        8024976 kB
MemFree:         6351656 kB
MemAvailable:    7396440 kB
Buffers:           92252 kB
Cached:          1083424 kB
SwapCached:            0 kB
Active:           550040 kB
Inactive:         901832 kB
Active(anon):     276808 kB
Inactive(anon):        0 kB
Active(file):     273232 kB
Inactive(file):   901832 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:       1048572 kB
SwapFree:        1048572 kB
Zswap:                 0 kB
Zswapped:              0 kB
Dirty:                56 kB
Writeback:             0 kB
AnonPages:        265204 kB
Mapped:           164804 kB
Shmem:               604 kB
KReclaimable:      32504 kB
Slab:              77944 kB
SReclaimable:      32504 kB
SUnreclaim:        45440 kB
KernelStack:        5088 kB
PageTables:         2844 kB
SecPageTables:         0 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     5061060 kB
Committed_AS:    1086040 kB
VmallocTotal:   135288315904 kB
VmallocUsed:        5208 kB
VmallocChunk:          0 kB
Percpu:             2680 kB
AnonHugePages:    180224 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
FileHugePages:         0 kB
FilePmdMapped:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB

/sys/kernel/mm/transparent_hugepage/enabled: [always] madvise never
/sys/kernel/mm/transparent_hugepage/hpage_pmd_size: 2097152
/sys/kernel/mm/transparent_hugepage/shmem_enabled: always within_size advise [never] deny force
/sys/kernel/mm/transparent_hugepage/defrag (defrag/compaction efforts parameter): always defer defer+madvise [madvise] never

Process Memory:
Virtual Size: 1815188K (peak: 1876072K)
Resident Set Size: 35096K (peak: 35096K) (anon: 20180K, file: 14916K, shmem: 0K)
Swapped out: 0K
C-Heap outstanding allocations: 6421K, retained: 502K
glibc malloc tunables: (default)

/proc/sys/kernel/threads-max (system-wide limit on the number of threads): 62645
/proc/sys/vm/max_map_count (maximum number of memory map areas a process may have): 262144
/proc/sys/vm/swappiness (control to define how aggressively the kernel swaps out anonymous memory): 60
/proc/sys/kernel/pid_max (system-wide limit on number of process identifiers): 99999

container information not found.
Steal ticks since vm start: 0
Steal ticks percentage since vm start:  0.000

CPU: total 10 (initial active 10) 0x61:0x0:0x000:0, fp, asimd, evtstrm, aes, pmull, sha1, sha256, crc32, lse, dcpop, sha3, sha512, paca, sve2
/proc/cpuinfo:
processor	: 0
BogoMIPS	: 48.00
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 asimddp sha512 asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 flagm2 frint svei8mm svebf16 bf16 afp
CPU implementer	: 0x61
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0x000
CPU revision	: 0

processor	: 1
BogoMIPS	: 48.00
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 asimddp sha512 asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 flagm2 frint svei8mm svebf16 bf16 afp
CPU implementer	: 0x61
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0x000
CPU revision	: 0

processor	: 2
BogoMIPS	: 48.00
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 asimddp sha512 asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 flagm2 frint svei8mm svebf16 bf16 afp
CPU implementer	: 0x61
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0x000
CPU revision	: 0

processor	: 3
BogoMIPS	: 48.00
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 asimddp sha512 asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 flagm2 frint svei8mm svebf16 bf16 afp
CPU implementer	: 0x61
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0x000
CPU revision	: 0

processor	: 4
BogoMIPS	: 48.00
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 asimddp sha512 asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 flagm2 frint svei8mm svebf16 bf16 afp
CPU implementer	: 0x61
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0x000
CPU revision	: 0

processor	: 5
BogoMIPS	: 48.00
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 asimddp sha512 asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 flagm2 frint svei8mm svebf16 bf16 afp
CPU implementer	: 0x61
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0x000
CPU revision	: 0

processor	: 6
BogoMIPS	: 48.00
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 asimddp sha512 asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 flagm2 frint svei8mm svebf16 bf16 afp
CPU implementer	: 0x61
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0x000
CPU revision	: 0

processor	: 7
BogoMIPS	: 48.00
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 asimddp sha512 asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 flagm2 frint svei8mm svebf16 bf16 afp
CPU implementer	: 0x61
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0x000
CPU revision	: 0

processor	: 8
BogoMIPS	: 48.00
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 asimddp sha512 asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 flagm2 frint svei8mm svebf16 bf16 afp
CPU implementer	: 0x61
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0x000
CPU revision	: 0

processor	: 9
BogoMIPS	: 48.00
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 asimddp sha512 asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 flagm2 frint svei8mm svebf16 bf16 afp
CPU implementer	: 0x61
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0x000
CPU revision	: 0


Online cpus: 0-9
Offline cpus: 

Memory: 4k page, physical 8024976k(7396440k free), swap 1048572k(1048572k free)
Page Sizes: 4k

vm_info: OpenJDK 64-Bit Server VM (23+37-2369) for linux-aarch64 JRE (23+37-2369), built on 2024-08-20T16:01:23Z with gcc 13.2.0

END.
2

What does your docker compose file look like?

3

@Christian_Dahlqvist
This is the file I downloaded from elasticsearch installation guide

it didn't work, so I have tried adding -

ES_JAVA_OPTS='-XX:UseSVE=0'
CLI_JAVA_OPTS='-XX:UseSVE=0'
and
platform: linux/arm64

I also tried to override ES_JAVA_HOME=/usr/share/elasticsearch/jdk

It's still not working


services:
  setup:
    platform: linux/arm64
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    volumes:
      - certs:/usr/share/elasticsearch/config/certs
    user: "0"
    command: >
      bash -c '
        if [ x${ELASTIC_PASSWORD} == x ]; then
          echo "Set the ELASTIC_PASSWORD environment variable in the .env file";
          exit 1;
        elif [ x${KIBANA_PASSWORD} == x ]; then
          echo "Set the KIBANA_PASSWORD environment variable in the .env file";
          exit 1;
        fi;
        if [ ! -f config/certs/ca.zip ]; then
          echo "Creating CA";
          bin/elasticsearch-certutil ca --silent --pem -out config/certs/ca.zip;
          unzip config/certs/ca.zip -d config/certs;
        fi;
        if [ ! -f config/certs/certs.zip ]; then
          echo "Creating certs";
          echo -ne \
          "instances:\n"\
          "  - name: es01\n"\
          "    dns:\n"\
          "      - es01\n"\
          "      - localhost\n"\
          "    ip:\n"\
          "      - 127.0.0.1\n"\
          "  - name: es02\n"\
          "    dns:\n"\
          "      - es02\n"\
          "      - localhost\n"\
          "    ip:\n"\
          "      - 127.0.0.1\n"\
          "  - name: es03\n"\
          "    dns:\n"\
          "      - es03\n"\
          "      - localhost\n"\
          "    ip:\n"\
          "      - 127.0.0.1\n"\
          > config/certs/instances.yml;
          bin/elasticsearch-certutil cert --silent --pem -out config/certs/certs.zip --in config/certs/instances.yml --ca-cert config/certs/ca/ca.crt --ca-key config/certs/ca/ca.key;
          unzip config/certs/certs.zip -d config/certs;
        fi;
        echo "Setting file permissions"
        chown -R root:root config/certs;
        find . -type d -exec chmod 750 \{\} \;;
        find . -type f -exec chmod 640 \{\} \;;
        echo "Waiting for Elasticsearch availability";
        until curl -s --cacert config/certs/ca/ca.crt https://es01:9200 | grep -q "missing authentication credentials"; do sleep 30; done;
        echo "Setting kibana_system password";
        until curl -s -X POST --cacert config/certs/ca/ca.crt -u "elastic:${ELASTIC_PASSWORD}" -H "Content-Type: application/json" https://es01:9200/_security/user/kibana_system/_password -d "{\"password\":\"${KIBANA_PASSWORD}\"}" | grep -q "^{}"; do sleep 10; done;
        echo "All done!";
      '
    healthcheck:
      test: ["CMD-SHELL", "[ -f config/certs/es01/es01.crt ]"]
      interval: 1s
      timeout: 5s
      retries: 120

  es01:
    depends_on:
      setup:
        condition: service_healthy
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    platform: linux/arm64
    volumes:
      - certs:/usr/share/elasticsearch/config/certs
      - esdata01:/usr/share/elasticsearch/data
    ports:
      - ${ES_PORT}:9200
    environment:
      - node.name=es01
      - cluster.name=${CLUSTER_NAME}
      - cluster.initial_master_nodes=es01,es02,es03
      - discovery.seed_hosts=es02,es03
      - ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
      - ES_JAVA_OPTS='-XX:UseSVE=0'
      - CLI_JAVA_OPTS='-XX:UseSVE=0'
      - bootstrap.memory_lock=true
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=certs/es01/es01.key
      - xpack.security.http.ssl.certificate=certs/es01/es01.crt
      - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.key=certs/es01/es01.key
      - xpack.security.transport.ssl.certificate=certs/es01/es01.crt
      - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.license.self_generated.type=${LICENSE}
      - xpack.ml.use_auto_machine_memory_percent=true
    mem_limit: ${MEM_LIMIT}
    ulimits:
      memlock:
        soft: -1
        hard: -1
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120

  es02:
    depends_on:
      - es01
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    platform: linux/arm64
    volumes:
      - certs:/usr/share/elasticsearch/config/certs
      - esdata02:/usr/share/elasticsearch/data
    environment:
      - node.name=es02
      - cluster.name=${CLUSTER_NAME}
      - cluster.initial_master_nodes=es01,es02,es03
      - discovery.seed_hosts=es01,es03
      - ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
      - ES_JAVA_OPTS='-XX:UseSVE=0'
      - CLI_JAVA_OPTS='-XX:UseSVE=0'
      - bootstrap.memory_lock=true
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=certs/es02/es02.key
      - xpack.security.http.ssl.certificate=certs/es02/es02.crt
      - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.key=certs/es02/es02.key
      - xpack.security.transport.ssl.certificate=certs/es02/es02.crt
      - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.license.self_generated.type=${LICENSE}
      - xpack.ml.use_auto_machine_memory_percent=true
    mem_limit: ${MEM_LIMIT}
    ulimits:
      memlock:
        soft: -1
        hard: -1
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120

  es03:
    depends_on:
      - es02
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    platform: linux/arm64
    volumes:
      - certs:/usr/share/elasticsearch/config/certs
      - esdata03:/usr/share/elasticsearch/data
    environment:
      - node.name=es03
      - cluster.name=${CLUSTER_NAME}
      - cluster.initial_master_nodes=es01,es02,es03
      - discovery.seed_hosts=es01,es02
      - ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
      - ES_JAVA_OPTS='-XX:UseSVE=0'
      - CLI_JAVA_OPTS='-XX:UseSVE=0'
      - bootstrap.memory_lock=true
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=certs/es03/es03.key
      - xpack.security.http.ssl.certificate=certs/es03/es03.crt
      - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.key=certs/es03/es03.key
      - xpack.security.transport.ssl.certificate=certs/es03/es03.crt
      - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.license.self_generated.type=${LICENSE}
      - xpack.ml.use_auto_machine_memory_percent=true
    mem_limit: ${MEM_LIMIT}
    ulimits:
      memlock:
        soft: -1
        hard: -1
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120

  kibana:
    depends_on:
      es01:
        condition: service_healthy
      es02:
        condition: service_healthy
      es03:
        condition: service_healthy
    image: docker.elastic.co/kibana/kibana:${STACK_VERSION}
    volumes:
      - certs:/usr/share/kibana/config/certs
      - kibanadata:/usr/share/kibana/data
    ports:
      - ${KIBANA_PORT}:5601
    environment:
      - SERVERNAME=kibana
      - ELASTICSEARCH_HOSTS=https://es01:9200
      - ELASTICSEARCH_USERNAME=kibana_system
      - ELASTICSEARCH_PASSWORD=${KIBANA_PASSWORD}
      - ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=config/certs/ca/ca.crt
    mem_limit: ${MEM_LIMIT}
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s -I http://localhost:5601 | grep -q 'HTTP/1.1 302 Found'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120

volumes:
  certs:
    driver: local
  esdata01:
    driver: local
  esdata02:
    driver: local
  esdata03:
    driver: local
  kibanadata:
    driver: local
4

What is MEM_LIMIT set to?

5

1073741824

6

3 ideas:

  1. The problem was reported with macOS 15.2. 15.3 is out, have you updated? (I dont know if it was fixed in 15.3, just asking to check)

  2. syntax is maybe wrong, try

      - 'ES_JAVA_OPTS=-XX:UseSVE=0'
      - 'CLI_JAVA_OPTS=-XX:UseSVE=0'

i.e. move the quotes

  1. Not sure why you are using platform: linux/arm64. On my M1 Mac I dont use that and my docker images run fine on aarch64. But I've not got a M4 Mac to check. You can check by opening terminal inside the container and doing a uname -a. You can also use ps -ef to see what's args are passed to java, also inside the container.

@Salil_Hardeniya also, the docker compose file you pasted above seems to have everything duplicated? Like you pasted it twice.

7

@RainTown Updated to 15.3 and tried things you mentioned. It's still not working. I even tried overriding the jdk to bundled jdk but to no avail

8

Mmm. Forget about overriding the jdk, back out whatever you used to do that.

When you say it isn’t working, can you be more precise? What’s the full error? Did you fix the docker-compose.yml file?

In error above was this:

Command Line: -Xms4m -Xmx64m -XX:+UseSerialGC -Dcli.name= -Dcli.script=bin/elasticsearch-certutil -Dcli.libs=modules/x-pack-core,modules/x-pack-security,lib/tools/security-cli -Des.path.home=/usr/share/elasticsearch -Des.path.conf=/usr/share/elasticsearch/config -Des.distribution.type=docker org.elasticsearch.launcher.CliToolLauncher ca --silent --pem -out config/certs/ca.zip

Notice how it does not contain -XX:UseSVE=0. if you use the correct settings in your docker-compose.yaml file, it should add these to the (bundled) java command, you can check here in docker desktop

Screenshot 2025-02-09 at 11.29.55
Screenshot 2025-02-09 at 11.29.55809×379 82.9 KB

9

it's always 1 container which is not running and causing issue
[+] Running 6/6
:heavy_check_mark: Network experiment_default Created 0.1s
✘ Container experiment-setup-1 Error 126.3s
:heavy_check_mark: Container experiment-es01-1 Created 0.0s
:heavy_check_mark: Container experiment-es02-1 Created 0.0s
:heavy_check_mark: Container experiment-es03-1 Created 0.0s
:heavy_check_mark: Container experiment-kibana-1 Created

I checked env in inspect for container "experiment-setup-1" and it does not have any field like you do
just this "Env": [ "PATH=/usr/share/elasticsearch/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "ELASTIC_CONTAINER=true", "SHELL=/bin/bash" ],

I do have these fields for other containers

			"xpack.license.self_generated.type=basic",
			"xpack.security.http.ssl.certificate=certs/es01/es01.crt",
			"xpack.security.http.ssl.key=certs/es01/es01.key",
			"ES_JAVA_OPTS='-XX:UseSVE=0'",
			"CLI_JAVA_OPTS='-XX:UseSVE=0'",
			"xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt",
			"xpack.security.transport.ssl.enabled=true",
			"node.name=es01",
			"discovery.seed_hosts=es02,es03",
			"xpack.security.http.ssl.enabled=true",
			"xpack.ml.use_auto_machine_memory_percent=true",
			"xpack.security.enabled=true",
			"cluster.initial_master_nodes=es01,es02,es03",
			"cluster.name=docker-cluster",
			"xpack.security.transport.ssl.verification_mode=certificate",
			"xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt",
			"xpack.security.transport.ssl.certificate=certs/es01/es01.crt",
			"bootstrap.memory_lock=true",
			"ELASTIC_PASSWORD=+m=4bnaW0lPa23NoXMnB",
			"xpack.security.transport.ssl.key=certs/es01/es01.key",
			"PATH=/usr/share/elasticsearch/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
			"ELASTIC_CONTAINER=true",
			"SHELL=/bin/bash"
		],

this is the error I'm getting,

2025-02-09 16:04:14 #
2025-02-09 16:04:14 # A fatal error has been detected by the Java Runtime Environment:
2025-02-09 16:04:14 #
2025-02-09 16:04:14 #  SIGILL (0x4) at pc=0x0000ffff87d400a8, pid=9, tid=18
2025-02-09 16:04:14 #
2025-02-09 16:04:14 # JRE version:  (23.0+37) (build )
2025-02-09 16:04:14 # Java VM: OpenJDK 64-Bit Server VM (23+37-2369, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, serial gc, linux-aarch64)
2025-02-09 16:04:14 # Problematic frame:
2025-02-09 16:04:14 # j  java.lang.System.registerNatives()V+0 java.base@23
2025-02-09 16:04:14 #
2025-02-09 16:04:14 # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
2025-02-09 16:04:14 #
2025-02-09 16:04:14 # An error report file with more information is saved as:
2025-02-09 16:04:14 # /usr/share/elasticsearch/hs_err_pid9.log
2025-02-09 16:04:14 [0.014s][warning][os] Loading hsdis library failed
2025-02-09 16:04:14 #
2025-02-09 16:04:14 # The crash happened outside the Java Virtual Machine in native code.
2025-02-09 16:04:14 # See problematic frame for where to report the bug.
2025-02-09 16:04:14 #
2025-02-09 16:04:14 Creating certs
2025-02-09 16:04:14 #
2025-02-09 16:04:14 # A fatal error has been detected by the Java Runtime Environment:
2025-02-09 16:04:14 #
2025-02-09 16:04:14 #  SIGILL (0x4) at pc=0x0000ffff9bd400a8, pid=22, tid=31
2025-02-09 16:04:14 #
2025-02-09 16:04:14 # JRE version:  (23.0+37) (build )
2025-02-09 16:04:14 # Java VM: OpenJDK 64-Bit Server VM (23+37-2369, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, serial gc, linux-aarch64)
2025-02-09 16:04:14 # Problematic frame:
2025-02-09 16:04:14 # j  java.lang.System.registerNatives()V+0 java.base@23
2025-02-09 16:04:14 #
2025-02-09 16:04:14 bash: line 12:     9 Aborted                 bin/elasticsearch-certutil ca --silent --pem -out config/certs/ca.zip
2025-02-09 16:04:14 unzip:  cannot find or open config/certs/ca.zip, config/certs/ca.zip.zip or config/certs/ca.zip.ZIP.
2025-02-09 16:04:14 bash: line 38:    22 Aborted                 bin/elasticsearch-certutil cert --silent --pem -out config/certs/certs.zip --in config/certs/instances.yml --ca-cert config/certs/ca/ca.crt --ca-key config/certs/ca/ca.key
2025-02-09 16:04:14 unzip:  cannot find or open config/certs/certs.zip, config/certs/certs.zip.zip or config/certs/certs.zip.ZIP.
2025-02-09 16:04:14 # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
2025-02-09 16:04:14 #
2025-02-09 16:04:14 # An error report file with more information is saved as:
2025-02-09 16:04:14 # /usr/share/elasticsearch/hs_err_pid22.log
2025-02-09 16:04:14 [0.008s][warning][os] Loading hsdis library failed
2025-02-09 16:04:14 #
2025-02-09 16:04:14 # The crash happened outside the Java Virtual Machine in native code.
2025-02-09 16:04:14 # See problematic frame for where to report the bug.
2025-02-09 16:04:14 #
2025-02-09 16:04:14 Setting file permissions
2025-02-09 16:04:15 Waiting for Elasticsearch availability

and below is my yml file

services:
  setup:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.17.1
    volumes:
      - certs:/usr/share/elasticsearch/config/certs
    user: "0"
    command: >
      bash -c '
        if [ x${ELASTIC_PASSWORD} == x ]; then
          echo "Set the ELASTIC_PASSWORD environment variable in the .env file";
          exit 1;
        elif [ x${KIBANA_PASSWORD} == x ]; then
          echo "Set the KIBANA_PASSWORD environment variable in the .env file";
          exit 1;
        fi;
        if [ ! -f config/certs/ca.zip ]; then
          echo "Creating CA";
          bin/elasticsearch-certutil ca --silent --pem -out config/certs/ca.zip;
          unzip config/certs/ca.zip -d config/certs;
        fi;
        if [ ! -f config/certs/certs.zip ]; then
          echo "Creating certs";
          echo -ne \
          "instances:\n"\
          "  - name: es01\n"\
          "    dns:\n"\
          "      - es01\n"\
          "      - localhost\n"\
          "    ip:\n"\
          "      - 127.0.0.1\n"\
          "  - name: es02\n"\
          "    dns:\n"\
          "      - es02\n"\
          "      - localhost\n"\
          "    ip:\n"\
          "      - 127.0.0.1\n"\
          "  - name: es03\n"\
          "    dns:\n"\
          "      - es03\n"\
          "      - localhost\n"\
          "    ip:\n"\
          "      - 127.0.0.1\n"\
          > config/certs/instances.yml;
          bin/elasticsearch-certutil cert --silent --pem -out config/certs/certs.zip --in config/certs/instances.yml --ca-cert config/certs/ca/ca.crt --ca-key config/certs/ca/ca.key;
          unzip config/certs/certs.zip -d config/certs;
        fi;
        echo "Setting file permissions"
        chown -R root:root config/certs;
        find . -type d -exec chmod 750 \{\} \;;
        find . -type f -exec chmod 640 \{\} \;;
        echo "Waiting for Elasticsearch availability";
        until curl -s --cacert config/certs/ca/ca.crt https://es01:9200 | grep -q "missing authentication credentials"; do sleep 30; done;
        echo "Setting kibana_system password";
        until curl -s -X POST --cacert config/certs/ca/ca.crt -u "elastic:${ELASTIC_PASSWORD}" -H "Content-Type: application/json" https://es01:9200/_security/user/kibana_system/_password -d "{\"password\":\"${KIBANA_PASSWORD}\"}" | grep -q "^{}"; do sleep 10; done;
        echo "All done!";
      '
    healthcheck:
      test: ["CMD-SHELL", "[ -f config/certs/es01/es01.crt ]"]
      interval: 1s
      timeout: 5s
      retries: 120

  es01:
    depends_on:
      setup:
        condition: service_healthy
    image: docker.elastic.co/elasticsearch/elasticsearch:8.17.1
    volumes:
      - certs:/usr/share/elasticsearch/config/certs
      - esdata01:/usr/share/elasticsearch/data
    ports:
      - ${ES_PORT}:9200
    environment:
      - node.name=es01
      - cluster.name=${CLUSTER_NAME}
      - cluster.initial_master_nodes=es01,es02,es03
      - discovery.seed_hosts=es02,es03
      - ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
      - 'ES_JAVA_OPTS=-XX:UseSVE=0'
      - 'CLI_JAVA_OPTS=-XX:UseSVE=0'
      - bootstrap.memory_lock=true
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=certs/es01/es01.key
      - xpack.security.http.ssl.certificate=certs/es01/es01.crt
      - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.key=certs/es01/es01.key
      - xpack.security.transport.ssl.certificate=certs/es01/es01.crt
      - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.license.self_generated.type=${LICENSE}
      - xpack.ml.use_auto_machine_memory_percent=true
    mem_limit: ${MEM_LIMIT}
    ulimits:
      memlock:
        soft: -1
        hard: -1
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120

  es02:
    depends_on:
      - es01
    image: docker.elastic.co/elasticsearch/elasticsearch:8.17.1
    volumes:
      - certs:/usr/share/elasticsearch/config/certs
      - esdata02:/usr/share/elasticsearch/data
    environment:
      - node.name=es02
      - cluster.name=${CLUSTER_NAME}
      - cluster.initial_master_nodes=es01,es02,es03
      - discovery.seed_hosts=es01,es03
      - ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
      - 'ES_JAVA_OPTS=-XX:UseSVE=0'
      - 'CLI_JAVA_OPTS=-XX:UseSVE=0'
      - bootstrap.memory_lock=true
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=certs/es02/es02.key
      - xpack.security.http.ssl.certificate=certs/es02/es02.crt
      - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.key=certs/es02/es02.key
      - xpack.security.transport.ssl.certificate=certs/es02/es02.crt
      - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.license.self_generated.type=${LICENSE}
      - xpack.ml.use_auto_machine_memory_percent=true
    mem_limit: ${MEM_LIMIT}
    ulimits:
      memlock:
        soft: -1
        hard: -1
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120

  es03:
    depends_on:
      - es02
    image: docker.elastic.co/elasticsearch/elasticsearch:8.17.1
    volumes:
      - certs:/usr/share/elasticsearch/config/certs
      - esdata03:/usr/share/elasticsearch/data
    environment:
      - node.name=es03
      - cluster.name=${CLUSTER_NAME}
      - cluster.initial_master_nodes=es01,es02,es03
      - discovery.seed_hosts=es01,es02
      - ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
      - 'ES_JAVA_OPTS=-XX:UseSVE=0'
      - 'CLI_JAVA_OPTS=-XX:UseSVE=0'
      - bootstrap.memory_lock=true
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=certs/es03/es03.key
      - xpack.security.http.ssl.certificate=certs/es03/es03.crt
      - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.key=certs/es03/es03.key
      - xpack.security.transport.ssl.certificate=certs/es03/es03.crt
      - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.license.self_generated.type=${LICENSE}
      - xpack.ml.use_auto_machine_memory_percent=true
    mem_limit: ${MEM_LIMIT}
    ulimits:
      memlock:
        soft: -1
        hard: -1
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120

  kibana:
    depends_on:
      es01:
        condition: service_healthy
      es02:
        condition: service_healthy
      es03:
        condition: service_healthy
    image: docker.elastic.co/kibana/kibana:${STACK_VERSION}
    volumes:
      - certs:/usr/share/kibana/config/certs
      - kibanadata:/usr/share/kibana/data
    ports:
      - ${KIBANA_PORT}:5601
    environment:
      - SERVERNAME=kibana
      - ELASTICSEARCH_HOSTS=https://es01:9200
      - ELASTICSEARCH_USERNAME=kibana_system
      - ELASTICSEARCH_PASSWORD=${KIBANA_PASSWORD}
      - ES_JAVA_HOME=/usr/share/elasticsearch/jdk
      - ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=config/certs/ca/ca.crt
    mem_limit: ${MEM_LIMIT}
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s -I http://localhost:5601 | grep -q 'HTTP/1.1 302 Found'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120

volumes:
  certs:
    driver: local
  esdata01:
    driver: local
  esdata02:
    driver: local
  esdata03:
    driver: local
  kibanadata:
    driver: local
10

Ok, you are mixing a few things up a bit here.

The setup container should start, wait for elasticsearch cluster to be up, first time round it will populate the certs volume, set the kibana_system password, and then exit. Thats all it does.

If you look in logs tab for setup, you will see something like

2025-02-09 13:22:41 Creating CA
2025-02-09 13:22:42 Archive:  config/certs/ca.zip
2025-02-09 13:22:42    creating: config/certs/ca/
2025-02-09 13:22:42   inflating: config/certs/ca/ca.crt  
2025-02-09 13:22:42   inflating: config/certs/ca/ca.key  
2025-02-09 13:22:42 Creating certs
2025-02-09 13:22:43 Archive:  config/certs/certs.zip
2025-02-09 13:22:43    creating: config/certs/es01/
2025-02-09 13:22:43   inflating: config/certs/es01/es01.crt  
2025-02-09 13:22:43   inflating: config/certs/es01/es01.key  
2025-02-09 13:22:43    creating: config/certs/es02/
2025-02-09 13:22:43   inflating: config/certs/es02/es02.crt  
2025-02-09 13:22:43   inflating: config/certs/es02/es02.key  
2025-02-09 13:22:43    creating: config/certs/es03/
2025-02-09 13:22:43   inflating: config/certs/es03/es03.crt  
2025-02-09 13:22:43   inflating: config/certs/es03/es03.key  
2025-02-09 13:22:43 Setting file permissions
2025-02-09 13:22:47 Waiting for Elasticsearch availability

Note the certs volume will persist, if its corrupted you need remove it and start again.

Your kibana section also has

  - ES_JAVA_HOME=/usr/share/elasticsearch/jdk

but this actually isn't needed as kibana is not a java app, please remove it.

Otherwise your yml file is IDENTICAL to mine. Here's my output, my file is called docker-compose.yaml:

[+] Running 11/11
 ✔ Network docker_default      Created                                                                             0.1s
 ✔ Volume "docker_esdata01"    Created                                                                             0.0s
 ✔ Volume "docker_esdata02"    Created                                                                             0.0s
 ✔ Volume "docker_esdata03"    Created                                                                             0.0s
 ✔ Volume "docker_kibanadata"  Created                                                                             0.0s
 ✔ Volume "docker_certs"       Created                                                                             0.0s
 ✔ Container docker-setup-1    Healthy                                                                             3.8s
 ✔ Container docker-es01-1     Healthy                                                                            34.5s
 ✔ Container docker-es02-1     Healthy                                                                            35.0s
 ✔ Container docker-es03-1     Healthy                                                                            34.9s
 ✔ Container docker-kibana-1   Started                                                                            35.0s

% docker ps
CONTAINER ID   IMAGE                                                  COMMAND                  CREATED         STATUS                   PORTS                              NAMES
98a1c761990a   docker.elastic.co/kibana/kibana:8.17.1                 "/bin/tini -- /usr/l…"   3 minutes ago   Up 2 minutes (healthy)   0.0.0.0:5601->5601/tcp             docker-kibana-1
85836069886a   docker.elastic.co/elasticsearch/elasticsearch:8.17.1   "/bin/tini -- /usr/l…"   3 minutes ago   Up 3 minutes (healthy)   9200/tcp, 9300/tcp                 docker-es03-1
6235a3a07fc9   docker.elastic.co/elasticsearch/elasticsearch:8.17.1   "/bin/tini -- /usr/l…"   3 minutes ago   Up 3 minutes (healthy)   9200/tcp, 9300/tcp                 docker-es02-1
da60e9928165   docker.elastic.co/elasticsearch/elasticsearch:8.17.1   "/bin/tini -- /usr/l…"   3 minutes ago   Up 3 minutes (healthy)   0.0.0.0:9200->9200/tcp, 9300/tcp   docker-es01-1

What's in you .env file?

What version of docker are you using?

The ES_JAVA_OPTS / CLI_JAVA_OPTS settings are only required within the es01/2/3 containers.

Ah, now I see , maybe not, add a section in the setup container like:

    environment:
      - 'ES_JAVA_OPTS=-XX:UseSVE=0'
      - 'CLI_JAVA_OPTS=-XX:UseSVE=0'

as the setup needs to run

bin/elasticsearch-certutil

which itself runs

bin/elasticsearch-cli

which itself is a java app, but it does use the CLI_JAVA_OPTS env variable.

Do you have any data in the cluster? if not, then I'd suggest remove all the containers and all the relevant volumes, and try again from scratch.

11

Below is my working docker-compose.yaml file. Its based on yours.

I added a sleep command to setup so the VM hangs around for an hour.

If you go into the exec tab of the setup container, and type

env

you should see

sh-5.0# env
SHELL=/bin/bash
HOSTNAME=35f0f92eeae1
ELASTIC_CONTAINER=true
PWD=/usr/share/elasticsearch
ES_JAVA_OPTS=-XX:UseSVE=0
HOME=/root
CLI_JAVA_OPTS=-XX:UseSVE=0
TERM=xterm
SHLVL=1
PATH=/usr/share/elasticsearch/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
_=/usr/bin/env
sh-5.0#

docker-compose.yaml

services:
  setup:
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    volumes:
      - certs:/usr/share/elasticsearch/config/certs
    environment:
      - 'ES_JAVA_OPTS=-XX:UseSVE=0'
      - 'CLI_JAVA_OPTS=-XX:UseSVE=0'
    user: "0"
    command: >
      bash -c '
        if [ x${ELASTIC_PASSWORD} == x ]; then
          echo "Set the ELASTIC_PASSWORD environment variable in the .env file";
          exit 1;
        elif [ x${KIBANA_PASSWORD} == x ]; then
          echo "Set the KIBANA_PASSWORD environment variable in the .env file";
          exit 1;
        fi;
        if [ ! -f config/certs/ca.zip ]; then
          echo "Creating CA";
          bin/elasticsearch-certutil ca --silent --pem -out config/certs/ca.zip;
          unzip config/certs/ca.zip -d config/certs;
        fi;
        if [ ! -f config/certs/certs.zip ]; then
          echo "Creating certs";
          echo -ne \
          "instances:\n"\
          "  - name: es01\n"\
          "    dns:\n"\
          "      - es01\n"\
          "      - localhost\n"\
          "    ip:\n"\
          "      - 127.0.0.1\n"\
          "  - name: es02\n"\
          "    dns:\n"\
          "      - es02\n"\
          "      - localhost\n"\
          "    ip:\n"\
          "      - 127.0.0.1\n"\
          "  - name: es03\n"\
          "    dns:\n"\
          "      - es03\n"\
          "      - localhost\n"\
          "    ip:\n"\
          "      - 127.0.0.1\n"\
          > config/certs/instances.yml;
          bin/elasticsearch-certutil cert --silent --pem -out config/certs/certs.zip --in config/certs/instances.yml --ca-cert config/certs/ca/ca.crt --ca-key config/certs/ca/ca.key;
          unzip config/certs/certs.zip -d config/certs;
        fi;
        echo "Setting file permissions"
        chown -R root:root config/certs;
        find . -type d -exec chmod 750 \{\} \;;
        find . -type f -exec chmod 640 \{\} \;;
        echo "Waiting for Elasticsearch availability";
        until curl -s --cacert config/certs/ca/ca.crt https://es01:9200 | grep -q "missing authentication credentials"; do sleep 30; done;
        echo "Setting kibana_system password";
        until curl -s -X POST --cacert config/certs/ca/ca.crt -u "elastic:${ELASTIC_PASSWORD}" -H "Content-Type: application/json" https://es01:9200/_security/user/kibana_system/_password -d "{\"password\":\"${KIBANA_PASSWORD}\"}" | grep -q "^{}"; do sleep 10; done;
        echo "All done!";
        echo "Will sleep for a bit ...";
        sleep 3600
      '
    healthcheck:
      test: ["CMD-SHELL", "[ -f config/certs/es01/es01.crt ]"]
      interval: 1s
      timeout: 5s
      retries: 120

  es01:
    depends_on:
      setup:
        condition: service_healthy
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    volumes:
      - certs:/usr/share/elasticsearch/config/certs
      - esdata01:/usr/share/elasticsearch/data
    ports:
      - ${ES_PORT}:9200
    environment:
      - node.name=es01
      - cluster.name=${CLUSTER_NAME}
      - cluster.initial_master_nodes=es01,es02,es03
      - discovery.seed_hosts=es02,es03
      - ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
      - 'ES_JAVA_OPTS=-XX:UseSVE=0'
      - 'CLI_JAVA_OPTS=-XX:UseSVE=0'
      - bootstrap.memory_lock=true
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=certs/es01/es01.key
      - xpack.security.http.ssl.certificate=certs/es01/es01.crt
      - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.key=certs/es01/es01.key
      - xpack.security.transport.ssl.certificate=certs/es01/es01.crt
      - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.license.self_generated.type=${LICENSE}
      - xpack.ml.use_auto_machine_memory_percent=true
    mem_limit: ${MEM_LIMIT}
    ulimits:
      memlock:
        soft: -1
        hard: -1
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120

  es02:
    depends_on:
      - es01
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    volumes:
      - certs:/usr/share/elasticsearch/config/certs
      - esdata02:/usr/share/elasticsearch/data
    environment:
      - node.name=es02
      - cluster.name=${CLUSTER_NAME}
      - cluster.initial_master_nodes=es01,es02,es03
      - discovery.seed_hosts=es01,es03
      - ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
      - 'ES_JAVA_OPTS=-XX:UseSVE=0'
      - 'CLI_JAVA_OPTS=-XX:UseSVE=0'
      - bootstrap.memory_lock=true
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=certs/es02/es02.key
      - xpack.security.http.ssl.certificate=certs/es02/es02.crt
      - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.key=certs/es02/es02.key
      - xpack.security.transport.ssl.certificate=certs/es02/es02.crt
      - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.license.self_generated.type=${LICENSE}
      - xpack.ml.use_auto_machine_memory_percent=true
    mem_limit: ${MEM_LIMIT}
    ulimits:
      memlock:
        soft: -1
        hard: -1
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120

  es03:
    depends_on:
      - es02
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    volumes:
      - certs:/usr/share/elasticsearch/config/certs
      - esdata03:/usr/share/elasticsearch/data
    environment:
      - node.name=es03
      - cluster.name=${CLUSTER_NAME}
      - cluster.initial_master_nodes=es01,es02,es03
      - discovery.seed_hosts=es01,es02
      - ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
      - 'ES_JAVA_OPTS=-XX:UseSVE=0'
      - 'CLI_JAVA_OPTS=-XX:UseSVE=0'
      - bootstrap.memory_lock=true
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=certs/es03/es03.key
      - xpack.security.http.ssl.certificate=certs/es03/es03.crt
      - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.key=certs/es03/es03.key
      - xpack.security.transport.ssl.certificate=certs/es03/es03.crt
      - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.license.self_generated.type=${LICENSE}
      - xpack.ml.use_auto_machine_memory_percent=true
    mem_limit: ${MEM_LIMIT}
    ulimits:
      memlock:
        soft: -1
        hard: -1
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120

  kibana:
    depends_on:
      es01:
        condition: service_healthy
      es02:
        condition: service_healthy
      es03:
        condition: service_healthy
    image: docker.elastic.co/kibana/kibana:${STACK_VERSION}
    volumes:
      - certs:/usr/share/kibana/config/certs
      - kibanadata:/usr/share/kibana/data
    ports:
      - ${KIBANA_PORT}:5601
    environment:
      - SERVERNAME=kibana
      - ELASTICSEARCH_HOSTS=https://es01:9200
      - ELASTICSEARCH_USERNAME=kibana_system
      - ELASTICSEARCH_PASSWORD=${KIBANA_PASSWORD}
      - ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=config/certs/ca/ca.crt
    mem_limit: ${MEM_LIMIT}
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s -I http://localhost:5601 | grep -q 'HTTP/1.1 302 Found'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120

volumes:
  certs:
    driver: local
  esdata01:
    driver: local
  esdata02:
    driver: local
  esdata03:
    driver: local
  kibanadata:
    driver: local
1 Like
12

@RainTown Thanks a lot, it worked.

I think adding this to the file worked.

environment:
      - 'ES_JAVA_OPTS=-XX:UseSVE=0'
      - 'CLI_JAVA_OPTS=-XX:UseSVE=0'

can you explain why you did this -
"I added a sleep command to setup so the VM hangs around for an hour."

Thanks a lot once again.

13

So, at first I thought the setup container just did some shell commands.

But then I realized it ran some other things which are actually little java applications (to setup the certs and populate that persistent volume).

This means it also needs the environment variables (it probably only needs one of them). Not just the es01/2/3 containers, the setup container too.

kibana container does not need the env variables as its not a java application.

I only added the sleep command so I could inspect the container while it was still running, check things, it's not actually required to get things working. But without it the setup container just exits. It will still exit in an hour if you just leave it, or you can take out the sleep 3600 from docker-compose.yaml

None of this env variable faffing will be necessary once the underlying bug is fixed.

I think key thing is the syntax here, with quotes in right place:

      - 'ES_JAVA_OPTS=-XX:UseSVE=0'
      - 'CLI_JAVA_OPTS=-XX:UseSVE=0'

By the way, for anyone at Elastic, the supplied docker compose file for the 3 node cluster sets the kibana_system password to .. whatever you set it to. I'm not sure what value that adds, as you can't do much with that kibana_system login. This might confuse a few people.

Screenshot 2025-02-09 at 14.19.32
Screenshot 2025-02-09 at 14.19.32532×309 25.1 KB

1 Like