Docker Logstash to Elastic: connection refused

[Adrien_Renaud]

I use the following official dockers:

docker run -d -p 9200:9200 -p 9300:9300 --name myel \
            -e "discovery.type=single-node" \
            -e "XPACK_SECURITY_ENABLED=false" \
            -e "XPACK_REPORTING_ENABLED=false" \
            -e "XPACK_MONITORING_ENABLED=false" \
            docker.elastic.co/elasticsearch/elasticsearch:6.2.3

docker run --name logstash --rm \
        -v $PWD/data:/usr/tmp \
        -v $PWD/logstash_confs/:/usr/share/logstash/pipeline/ \
        -e "ELASTIC_HOST=localhost:9200" \
        -e "XPACK_SECURITY_ENABLED=false" \
        -e "XPACK_REPORTING_ENABLED=false" \
        -e "XPACK_MONITORING_ENABLED=false" \
        docker.elastic.co/logstash/logstash:6.2.3

And there is my logstash conf:

input {
    file {
        path => "/usr/tmp/ts_indicator_description.csv"
        start_position => "beginning"
        sincedb_path => "/dev/null"
    }
}

filter {
  csv {
    separator => ","
    #autodetect_column_names => "true"
    columns => ['indicator_name', 'indicator_description']
    skip_header => "true"
  }
}

output {
  elasticsearch {
    hosts => "${ELASTIC_HOST}"
    user => ""
    password => ""
    index => "ts_indicator_description"
  }
  #stdout { codec => rubydebug }
}

But I'm getting this explicit error message:
[2018-06-12T18:13:37,724][WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://localhost:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketException] Connection refused (Connection refused)"}

And indeed login to the logstash container:

$ sudo docker exec -it mylo bash
bash-4.2$ curl -X GET localhost:9200
curl: (7) Failed connect to localhost:9200; Connection refused

But from the host, curl -X GET localhost:9200 is giving me the now famous "You know, for search"

Can we solve this?

[jakelandis]

Try adding a --net="host" to your docker run arguments.

docker run --net="host" --name logstash --rm \
        -v $PWD/data:/usr/tmp \
        -v $PWD/logstash_confs/:/usr/share/logstash/pipeline/ \
        -e "ELASTIC_HOST=localhost:9200" \
        -e "XPACK_SECURITY_ENABLED=false" \
        -e "XPACK_REPORTING_ENABLED=false" \
        -e "XPACK_MONITORING_ENABLED=false" \
        docker.elastic.co/logstash/logstash:6.2.3

[Adrien_Renaud]

Thank you very much. Adding --net=host did the trick. For posterity, here is my commands to run ELK:

docker run --name mylo \
       --net=host\
        -v $PWD/data:/usr/tmp \
        -v $PWD/logstash_confs/:/usr/share/logstash/pipeline/ \
        -e "ELASTIC_HOST=localhost:9200" \
        -e "XPACK_SECURITY_ENABLED=false" \
        -e "XPACK_REPORTING_ENABLED=false" \
        -e "XPACK_MONITORING_ENABLED=false" \
        docker.elastic.co/logstash/logstash:6.2.3


docker run -d -p 9200:9200 -p 9300:9300 --name myel \
                --net=host\
                -e "discovery.type=single-node" \
                -e "XPACK_SECURITY_ENABLED=false" \
                -e "XPACK_REPORTING_ENABLED=false" \
                -e "XPACK_MONITORING_ENABLED=false" \
                docker.elastic.co/elasticsearch/elasticsearch:6.2.3


docker run --name myki -p 5601:5601 -d \
                --net=host \
                -e "ELASTICSEARCH_URL=http://localhost:9200" \
                docker.elastic.co/kibana/kibana:6.2.3