ConnectionError: socket hang up

I successfully run this docker compose locally on macOS and can work with ELK, but when running the same compose I encounter the following errors:

[2024-01-13T14:26:44.001+00:00][ERROR][plugins.security.authorization] Error registering Kibana Privileges with Elasticsearch for kibana-.kibana: connect ECONNREFUSED 172.18.0.3:9200

And the consequence of this is most likely this error:

{"statusCode":503,"error":"Service Unavailable","message":"License is not available."}

I'm suggesting that the problem might be in IPTABLES, but since I'm new to this technology, I can't figure out what exactly I need to do.

But I see that no packets have been dropped in the DOCKER-ISOLATION-STAGE-2 chain, which makes me confused in my opinion about IPTABLES.

> Chain DOCKER-ISOLATION-STAGE-2 (2 references)
> num   pkts bytes target     prot opt in     out     source               destination         
> 1        0     0 DROP       all  --  *      br-ba4e335d8552  0.0.0.0/0            0.0.0.0/0           
> 2        0     0 DROP       all  --  *      docker0  0.0.0.0/0            0.0.0.0/0           
> 3      111 22701 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Please tell me how can I fix this to run this compose successfully on ubuntu?

Here is the full Kibana log obtained with grep ERROR:

[2024-01-13T14:26:43.966+00:00][ERROR][plugins.securitySolution.endpoint:user-artifact-packager:1.0.0] EndpointError: Error scheduling task, received socket hang up - Local: 172.18.0.4:38172, Remote: unknown:unknown
[2024-01-13T14:26:43.968+00:00][ERROR][plugins.taskManager] Task cases-telemetry-task "cases-telemetry-task" failed: ConnectionError: socket hang up - Local: 172.18.0.4:38102, Remote: 172.18.0.3:9200
[2024-01-13T14:26:43.970+00:00][ERROR][plugins.taskManager] Task Fleet-Usage-Sender "Fleet-Usage-Sender-1.1.0" failed: ConnectionError: read ECONNRESET - Local: unknown:unknown, Remote: unknown:unknown
[2024-01-13T14:26:44.001+00:00][ERROR][plugins.security.authorization] Error registering Kibana Privileges with Elasticsearch for kibana-.kibana: connect ECONNREFUSED 172.18.0.3:9200
[2024-01-13T14:26:44.177+00:00][ERROR][plugins.security.authorization] Error registering Kibana Privileges with Elasticsearch for kibana-.kibana: getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:26:44.186+00:00][ERROR][plugins.fleet] Failed to fetch latest version of synthetics from registry: '403 Forbidden' error response from package registry at https://epr.elastic.co/search?package=synthetics&prerelease=true&kibana.version=8.9.1
[2024-01-13T14:26:44.247+00:00][ERROR][plugins.fleet] uninstalling synthetics-1.0.1 after error installing: [ConnectionError: getaddrinfo EAI_AGAIN elasticsearch]
[2024-01-13T14:26:44.251+00:00][ERROR][plugins.fleet] failed to update package status to: install_failed  ConnectionError: getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:26:44.252+00:00][ERROR][plugins.fleet] failed to uninstall or rollback package after installation error Error: synthetics is not installed
[2024-01-13T14:26:44.252+00:00][ERROR][plugins.synthetics] ConnectionError: Error installing synthetics 1.0.1: getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:26:44.416+00:00][ERROR][plugins.security.authorization] Error registering Kibana Privileges with Elasticsearch for kibana-.kibana: getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:26:44.648+00:00][ERROR][plugins.taskManager] Task security:endpoint-diagnostics "security:endpoint-diagnostics:1.0.0" failed: ConnectionError: getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:26:44.649+00:00][ERROR][plugins.taskManager] Task security:endpoint-meta-telemetry "security:endpoint-meta-telemetry:1.0.0" failed: ConnectionError: getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:26:44.800+00:00][ERROR][plugins.security.authorization] Error registering Kibana Privileges with Elasticsearch for kibana-.kibana: getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:26:44.802+00:00][ERROR][elasticsearch-service] Unable to retrieve version information from Elasticsearch nodes. getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:26:47.798+00:00][ERROR][plugins.alerting] Error installing index template .alerts-security.alerts-default-index-template - getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:26:47.798+00:00][ERROR][plugins.alerting] ConnectionError: getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:26:47.799+00:00][ERROR][plugins.alerting] Error initializing context security - Failure during installation. getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:26:51.210+00:00][ERROR][plugins.alerting] Error installing index template .alerts-observability.logs.alerts-default-index-template - getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:26:51.211+00:00][ERROR][plugins.alerting] ConnectionError: getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:26:51.212+00:00][ERROR][plugins.alerting] Error initializing context observability.logs - Failure during installation. getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:26:52.223+00:00][ERROR][plugins.alerting] Error creating concrete write index - getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:26:52.223+00:00][ERROR][plugins.alerting] ConnectionError: getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:26:52.224+00:00][ERROR][plugins.alerting] Error initializing context observability.uptime - Failure during installation. getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:26:52.836+00:00][ERROR][plugins.alerting] Error creating concrete write index - getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:26:52.836+00:00][ERROR][plugins.alerting] ConnectionError: getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:26:52.837+00:00][ERROR][plugins.alerting] Error initializing context observability.slo - Failure during installation. getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:26:54.416+00:00][ERROR][plugins.alerting] Error installing index template .alerts-observability.metrics.alerts-default-index-template - getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:26:54.417+00:00][ERROR][plugins.alerting] ConnectionError: getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:26:54.418+00:00][ERROR][plugins.alerting] Error initializing context observability.metrics - Failure during installation. getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:26:55.047+00:00][ERROR][plugins.alerting] Error creating concrete write index - getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:26:55.047+00:00][ERROR][plugins.alerting] ConnectionError: getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:26:55.048+00:00][ERROR][plugins.alerting] Error initializing context observability.apm - Failure during installation. getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:27:13.951+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:27:24.052+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:27:26.646+00:00][ERROR][plugins.eventLog] error initializing elasticsearch resources: error checking existance of ilm policy: getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:27:26.647+00:00][ERROR][plugins.eventLog] initialization failed, events will not be indexed
[2024-01-13T14:27:34.135+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:27:44.208+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:27:54.275+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:28:04.366+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:28:14.441+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:28:24.537+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:28:34.650+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:28:44.744+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:28:54.816+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:29:04.888+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:29:14.960+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:29:25.055+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:29:35.143+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:29:45.228+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:29:55.308+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:30:05.382+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:30:15.456+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:30:25.539+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:30:35.612+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:30:45.700+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:30:55.769+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:31:05.850+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:31:15.921+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:31:26.010+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:31:36.103+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2024-01-13T14:31:38.140+00:00][ERROR][savedobjects-service.repository.point-in-time-finder] Failed to open PIT for types [application_usage_totals]
ConnectionError: getaddrinfo EAI_AGAIN elasticsearch
[2024-01-13T14:31:38.143+00:00][ERROR][savedobjects-service.repository.point-in-time-finder] Failed to open PIT for types [application_usage_daily]
    at KibanaTransport.request (/usr/share/kibana/node_modules/@elastic/transport/lib/Transport.js:528:31)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at KibanaTransport.request (/usr/share/kibana/node_modules/@kbn/core-elasticsearch-client-server-internal/src/create_transport.js:51:16)
    at ClientTraced.GetApi [as get] (/usr/share/kibana/node_modules/@elastic/elasticsearch/lib/api/api/get.js:36:12) {
  meta: {
    body: undefined,
    statusCode: 0,
    headers: {},
    meta: {
      context: null,
      request: [Object],
      name: 'elasticsearch-js',
      connection: [Object],
      attempts: 0,
      aborted: false
    },
    warnings: [Getter]
  },
  isBoom: true,
  isServer: true,
  data: null,
  output: {
    statusCode: 503,
    payload: {
      statusCode: 503,
      error: 'Service Unavailable',
      message: 'getaddrinfo EAI_AGAIN elasticsearch'
    },
    headers: {}
  },
  [Symbol(SavedObjectsClientErrorCode)]: 'SavedObjectsClient/esUnavailable'
}

Your docker-compose looks very similar to one in our documentation. I would suggest starting with that one exactly as-is, and then gradually customize it as needed.

See: Install Elasticsearch with Docker | Elasticsearch Guide [8.12] | Elastic
And: elasticsearch/docs/reference/setup/install/docker/docker-compose.yml at 8.12 · elastic/elasticsearch · GitHub

Yes, this is the official compose from Elastic, adapted to my needs, that’s why I asked for help here. Unfortunately, this does not answer my question.

Are you able to run all 3 nodes and get a Kibana connection using the "unadapted" docker compose configuration?

I took the original compose and env provided by Elastic:
compose: https://raw.githubusercontent.com/elastic/elasticsearch/8.12/docs/reference/setup/install/docker/docker-compose.yml
env:
https://raw.githubusercontent.com/elastic/elasticsearch/8.12/docs/reference/setup/install/docker/.env

I specified 8.9.1 as the version.

Here's the same error:

{"statusCode":503,"error":"Service Unavailable","message":"License is not available."}

For convenience, I have included the full log of the docker-compose.yml up command in my repository: