EntitlementBootstrap failure preventing startup: AttachNotSupportedException: Unable to open socket file

Elastic Stack Elasticsearch
50

sorry, too many posts, but looking in JDK source code ./src/hotspot/os/posix/attachListener_posix.cpp

is this:

  os::snprintf_checked(fn, sizeof(fn), ".attach_pid%d",
                       os::current_process_id());
  RESTARTABLE(::stat(fn, &st), ret);
  if (ret == -1) {
    log_trace(attach)("Failed to find attach file: %s, trying alternate", fn);
    os::snprintf_checked(fn, sizeof(fn), "%s/.attach_pid%d", os::get_temp_directory(),
                         os::current_process_id());
    RESTARTABLE(::stat(fn, &st), ret);
    if (ret == -1) {
      log_debug(attach)("Failed to find attach file: %s", fn);
    }
  }
  if (ret == 0) {
    // simple check to avoid starting the attach mechanism when
    // a bogus non-root user creates the file
    if (os::Posix::matches_effective_uid_or_root(st.st_uid)) {
      init();
      log_trace(attach)("Attach triggered by %s", fn);
      return true;
    } else {
      log_debug(attach)("File %s has wrong user id %d (vs %d). Attach is not triggered", fn, st.st_uid, geteuid());
    }
  }
  return false;

ret = 0 means (I think) the .attach_pid file was successfully created in the path.logs defined directory, which your logs also indicate is so. It then does an ownership checks. If debug logging was enabled for this specific part of the JDK, and I dont know how to do that, I think you may see the error there, as if I’ve understood correctly the files have different owners as the effective UID running elasticsearch, nor is that the root user.

EDIT you can see these logs with -Xlog:attach=trace added to jdk command line. In my system with 8.18.0 you see:

Oct 25 12:36:16 u2024 systemd-entrypoint[265689]: [3.369s][trace][attach] Failed to find attach file: .attach_pid265689, trying alternate
Oct 25 12:36:16 u2024 systemd-entrypoint[265689]: [3.369s][trace][attach] Attach triggered by /tmp/.attach_pid265689

This is because ti tried the “cwd” first, fails, then tries /tmp/…

In 8.19.5 the log is:

Oct 25 12:38:27 u2024 systemd-entrypoint[271234]: [3.054s][trace][attach] Attach triggered by .attach_pid271234
Oct 25 12:38:27 u2024 systemd-entrypoint[271234]: [3.054s][debug][attach] default streaming output: 1
Oct 25 12:38:27 u2024 systemd-entrypoint[271234]: [3.155s][debug][attach] read request: cmd = getversion
Oct 25 12:38:27 u2024 systemd-entrypoint[271234]: [3.155s][debug][attach] read request: arg = options
Oct 25 12:38:27 u2024 systemd-entrypoint[271234]: [3.155s][debug][attach] read request: arg =
Oct 25 12:38:27 u2024 systemd-entrypoint[271234]: [3.155s][debug][attach] read request: arg =
Oct 25 12:38:27 u2024 systemd-entrypoint[271234]: [3.155s][debug][attach] executing command getversion, streaming output: 1 (supported by impl: 1)
Oct 25 12:38:27 u2024 systemd-entrypoint[271234]: [3.156s][debug][attach] v2 request, data size = 195
Oct 25 12:38:27 u2024 systemd-entrypoint[271234]: [3.156s][debug][attach] option: streaming, value: 0
Oct 25 12:38:27 u2024 systemd-entrypoint[271234]: [3.156s][debug][attach] read request: cmd = load
Oct 25 12:38:27 u2024 systemd-entrypoint[271234]: [3.156s][debug][attach] read request: arg = instrument
Oct 25 12:38:27 u2024 systemd-entrypoint[271234]: [3.156s][debug][attach] read request: arg = false
Oct 25 12:38:27 u2024 systemd-entrypoint[271234]: [3.156s][debug][attach] read request: arg = /usr/share/elasticsearch/lib/entitlement-agent/elasticsearch-entitlement-agent-8.19.6.jar=org.elasticsearch.entitlement.initialization.EntitlementInitialization
Oct 25 12:38:27 u2024 systemd-entrypoint[271234]: [3.156s][debug][attach] executing command load, streaming output: 0 (supported by impl: 1

Here the CWD is /var/log/elasticsearch, as 8.19.5 chdir-ed to that directory (I am still unconvinced that was good idea) and obviously with my completely standard setup stuff works first time there, the .attach_pid* file is created there, and the Attach Listener is triggered.

In your case @buitcj I think you might see different messages with -Xlog:attach=trace - please try.

1 Like