[Elasticsearch Docker] [Errno 111] Connection refused when connecting with python script

Elastic Stack Elasticsearch
1

Hey,

I got an issue with a dockerized elasticsearch cluster that I would like to connect with a simple python indexing script.

After executing docker-compose up it brings up the cluster, kibana as well, but when it tries to connect to ES via the python script it get the following error:

elasticsearch.exceptions.ConnectionError: ConnectionError(<urllib3.connection.HTTPConnection object at 0x7f1c04e0c0d0>: Failed to establish a new connection: [Errno 111] Connection refused) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0x7f1c04e0c0d0>: Failed to establish a new connection: [Errno 111] Connection refused)

I can reach ES with the browser, I can also index regularly if I execute the script in pycharm. So it has to do with docker, but I can't find the source of the problem.
This is my dockerfile:

    version: '3.4'
services:
  es01:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.8.1
    container_name: es01
    environment:
      #- discovery.type=single-node needed?
      - node.name=es01
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es02,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - xpack.security.enabled=false
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data01:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
    networks:
      - elastic

  es02:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.8.1
    container_name: es02
    environment:
      - node.name=es02
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es01,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data02:/usr/share/elasticsearch/data
    networks:
      - elastic

  es03:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.8.1
    container_name: es03
    environment:
      - node.name=es03
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es01,es02
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data03:/usr/share/elasticsearch/data
    networks:
      - elastic

  kib01:
    image: docker.elastic.co/kibana/kibana:7.8.1
    container_name: kib01
    depends_on:
      - es01
      - es02
      - es03
    ports:
      - 5601:5601
    environment:
      ELASTICSEARCH_URL: http://es01:9200
      ELASTICSEARCH_HOSTS: http://es01:9200
    networks:
      - elastic

  web:
    build: .
    ports:
      - 8000:8000
    depends_on:
      - es01
      - es02
      - es03
    networks:
      - elastic


volumes:
  data01:
    driver: local
  data02:
    driver: local
  data03:
    driver: local

networks:
  elastic:
    driver: bridge

And the python script:

from elasticsearch import Elasticsearch, helpers
import sys, json, os
import requests

es = Elasticsearch(hosts=[{"host": 'localhost'}])

def load_json(directory):
    " Use a generator, no need to load all in memory"
    for filename in os.listdir(directory):
        if filename.endswith('.json'):
            filename = "JSON/" + filename
            with open(filename,'r') as open_file:
                yield json.load(open_file)

if __name__ == '__main__':
    helpers.bulk(es, load_json("JSON"), index='urteile')

Anybody knows what could be the problem here?

2

Can you just try

es = Elasticsearch()

and see if that works? Also is running curl/wget against localhost:9200 returning the expected results?

3

Tried it, didn't work unfortunately - still the same error. Curl localhost:9200 works as it should.

4

A curl command within the docker file also gives me a Connection refused by the way. This is the error I get:

mapping_1  | * TCP_NODELAY set
mapping_1  | * connect to 127.0.0.1 port 9200 failed: Connection refused
mapping_1  | *   Trying ::1...
mapping_1  | * TCP_NODELAY set
mapping_1  | * Immediate connect fail for ::1: Address not available
mapping_1  | *   Trying ::1...
mapping_1  | * TCP_NODELAY set
mapping_1  | * Immediate connect fail for ::1: Address not available
mapping_1  | * Failed to connect to localhost port 9200: Connection refused
mapping_1  | * Closing connection 0
mapping_1  | curl: (7) Failed to connect to localhost port 9200: Connection refused
5

Any new ideas on this? Couldn't resolve the issue so far...

6

If you are running your python script also within docker, then using localhost won't work, but you need to specify the elasticsearch host.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.