Elasticsearch Unreachable - Docker

Elastic Stack Elasticsearch
1

Hi!

I'm using the ELK Stack with Docker.
Everything worked fine, except the day when my server ran out of space because I didn't manage well the lifetime of my data.
I had to remove and build again my Docker containers.
My problem is that, I don't know if it's directly related or not, but I have now errors when I try to start Kibana and Logstash. They can't reach Elasticsearch service.

Errors:

logstash_1       | [2019-08-09T08:01:50,916][WARN ][logstash.licensechecker.licensereader] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://elastic:xxxxxx@elasticsearch:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://elastic:xxxxxx@elasticsearch:9200/][Manticore::ConnectTimeout] connect timed out"}
kibana_1         | {"type":"log","@timestamp":"2019-08-09T08:06:30Z","tags":["license","warning","xpack"],"pid":1,"message":"License information from the X-Pack plugin could not be obtained from Elasticsearch for the [data] cluster. Error: Request Timeout after 30000ms"}

However, Elasticsearch seems to be up:

curl -GET 'http://elastic:XXXXX@localhost:9200'
{
  "name" : "4bc961f74ca3",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "N1HmOLw4Q3umF8vMPiVQbA",
  "version" : {
    "number" : "7.2.1",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "fe6cb20",
    "build_date" : "2019-07-24T17:58:29.979462Z",
    "build_snapshot" : false,
    "lucene_version" : "8.0.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Here my config (come from https://github.com/deviantony/docker-elk):

docker-compose.yml:

version: '2'

services:

          elasticsearch:
            build:
              context: elasticsearch/
              args:
                ELK_VERSION: $ELK_VERSION
            volumes:
              - ./elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml:ro
            ports:
              - "9200:9200"
              - "9300:9300"
            environment:
              ES_JAVA_OPTS: "-Xmx512m -Xms512m"
              ELASTIC_PASSWORD: XXXXX
            networks:
              - elk

          logstash:
            build:
              context: logstash/
              args:
                ELK_VERSION: $ELK_VERSION
            volumes:
              - ./logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml:ro
              - ./logstash/pipeline:/usr/share/logstash/pipeline:ro
            ports:
              - "5000:5000"
              - "9600:9600"
            environment:
              LS_JAVA_OPTS: "-Xmx256m -Xms256m"
            networks:
              - elk
            depends_on:
              - elasticsearch

      kibana:
        build:
          context: kibana/
          args:
            ELK_VERSION: $ELK_VERSION
        volumes:
          - ./kibana/config/kibana.yml:/usr/share/kibana/config/kibana.yml:ro
        ports:
          - "5601:5601"
        networks:
          - elk
        depends_on:
          - elasticsearch

networks:

  elk:
    driver: bridge

elasticsearch.yml:

cluster.name: "docker-cluster"
network.host: 0.0.0.0

discovery.type: single-node

xpack.license.self_generated.type: basic
xpack.security.enabled: true
xpack.monitoring.collection.enabled: true

kibana.yml:

server.name: kibana
server.host: "0"
elasticsearch.hosts: [ "http://elasticsearch:9200" ]
xpack.monitoring.ui.container.elasticsearch.enabled: true

elasticsearch.username: elastic
elasticsearch.password: XXXXX

logstash.yml:

http.host: "0.0.0.0"
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.hosts: [ "http://elasticsearch:9200" ]
xpack.monitoring.elasticsearch.username: elastic
xpack.monitoring.elasticsearch.password: XXXXX

I'm pretty sure it's a basic mistake I made, but I checked everything and I don't see where it comes from.
It's strange because I didn't change any configuration since when it worked. I even put back the default setting from the git repository, and I have the same error.
Maybe when my server ran out of memory, it changes some settings on the server that I'm not aware? (in this case, it's not related to Elastic but I don't know where to search).

Thank you very much!

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.