# Dockerizing a working ELK project, 'Dead ES instance' **URL:** \ **Category:** Logstash\ **Created:** [August 4, 2020, 2:41am UTC](https://discuss.elastic.co/t/dockerizing-a-working-elk-project-dead-es-instance/243639 "2020-08-04T02:41:16Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![kl29](https://avatars.discourse-cdn.com/v4/letter/k/c57346/32.png) [@kl29](https://discuss.elastic.co/u/kl29)\ **Post date:** [August 4, 2020, 2:41am UTC](https://discuss.elastic.co/t/dockerizing-a-working-elk-project-dead-es-instance/243639/1 "2020-08-04T02:41:16Z")
My setup works without Dockerizing. Now, making a Dockerized project, logstash cannot connect to the ES instance. I set **http.host: "0.0.0.0"** in logstash.yml, and my pipeline.conf is: ```auto input { exec { command => "locale charmap" interval => 3600 } tcp { host => "0.0.0.0" port => "5043" codec => plain { charset => "ISO-8859-1" # "UTF-8" } } } filter { grok { match => { "message" => "(?\[(.*?)\])" } } if "_grokparsefailure" in [tags] { drop { } } } output { elasticsearch { hosts => ['elasticsearch'] index => "logstash-tcp-call-%{+YYYY.MM.dd}" } stdout{ codec => rubydebug } } ``` Maybe my host / dynamic routing is the issue, but I have tried the following possibilities: ['elasticsearch'] ['elasticsearch:9200'] ['192.168.1.7:9200'] ['0.0.0.0:9200'] Would really appreciate helping my Logstash connect to ES! For some reason it says: ```auto logstash | [WARN] 2020-08-04 01:59:58.622 [LogStash::Runner] licensereader - Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://0.0.0.0:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://0.0.0.0:9200/][Manticore::SocketException] Connection refused (Connection refused)"} ``` ---
**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\ **Post date:** [August 4, 2020, 11:47am UTC](https://discuss.elastic.co/t/dockerizing-a-working-elk-project-dead-es-instance/243639/2 "2020-08-04T11:47:18Z")
What does your Elasticsearch configuration look like? What do the logs of ES say? (The bootstrap checks might have failed.) And just to understand your infrastructure: Logstash and ES are on two different servers in the same network and one is called elasticsearch? ---
**Author:** ![kl29](https://avatars.discourse-cdn.com/v4/letter/k/c57346/32.png) [@kl29](https://discuss.elastic.co/u/kl29)\ **Post date:** [August 4, 2020, 3:20pm UTC](https://discuss.elastic.co/t/dockerizing-a-working-elk-project-dead-es-instance/243639/3 "2020-08-04T15:20:34Z")
> [@Jenni](#): > > **What does your Elasticsearch configuration look like? What do the logs of ES say? (The bootstrap checks might have failed.)** > **And just to understand your infrastructure: Logstash and ES are on two different servers in the same network and one is called elasticsearch?** Thank you for your prompt reply! **What does your Elasticsearch configuration look like?** So I have no explicit elasticsearch.yml, I have the following two yml: logstash.yml: ```auto http.host: "0.0.0.0" xpack.monitoring.elasticsearch.hosts: "http://elasticsearch:9200" ``` pipelines.yml: ```auto - pipeline.id: main path.config: "/usr/share/logstash/pipeline" ``` **What do the logs of ES say? (The bootstrap checks might have failed.)** I will followup a reply with the log. (didn't see any failed bootstrap tests) **And just to understand your infrastructure: Logstash and ES are on two different servers in the same network and one is called elasticsearch?** That sounds accurate; to confirm, here is my docker-compose.yml: ```auto version: "2.4" services: elasticsearch: image: elasticsearch:7.8.0 container_name: elasticsearch restart: always mem_reservation: 4000m mem_limit: 6000m environment: - discovery.type=single-node - MAX_MAP_COUNT=262144 volumes: - ./elasticsearch/data:/usr/share/elasticsearch/data ports: - "9200:9200" - "9300:9300" logstash: image: docker.elastic.co/logstash/logstash:7.8.0 container_name: logstash environment: - ELASTICSEARCH_HOSTS=http://elasticsearch:9200 restart: always mem_reservation: 2048m mem_limit: 4096m volumes: - ./logstash/pipeline:/usr/share/logstash/pipeline/ - ./logstash/config/:/usr/share/logstash/config/ ports: - "9600:9600" kibana: image: docker.elastic.co/kibana/kibana:7.8.0 container_name: kibana environment: - ELASTICSEARCH_HOSTS=http://elasticsearch:9200 restart: always ports: - "5601:5601" ``` ---
**Author:** ![kl29](https://avatars.discourse-cdn.com/v4/letter/k/c57346/32.png) [@kl29](https://discuss.elastic.co/u/kl29)\ **Post date:** [August 4, 2020, 3:21pm UTC](https://discuss.elastic.co/t/dockerizing-a-working-elk-project-dead-es-instance/243639/4 "2020-08-04T15:21:42Z")
```auto logstash | [INFO] 2020-08-04 15:17:48.633 [monitoring-license-manager] internalpipelinesource - Monitoring License OK logstash | [INFO] 2020-08-04 15:17:48.633 [monitoring-license-manager] internalpipelinesource - Validated license for monitoring. Enabling monitoring pipeline. logstash | [INFO] 2020-08-04 15:17:49.656 [[.monitoring-logstash]-pipeline-manager] elasticsearchmonitoring - Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://elasticsearch:9200/]}} logstash | [WARN] 2020-08-04 15:17:49.673 [[.monitoring-logstash]-pipeline-manager] elasticsearchmonitoring - Restored connection to ES instance {:url=>"http://elasticsearch:9200/"} logstash | [INFO] 2020-08-04 15:17:49.688 [[.monitoring-logstash]-pipeline-manager] elasticsearchmonitoring - ES Output version determined {:es_version=>7} logstash | [WARN] 2020-08-04 15:17:49.688 [[.monitoring-logstash]-pipeline-manager] elasticsearchmonitoring - Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7} logstash | [INFO] 2020-08-04 15:17:49.713 [[.monitoring-logstash]-pipeline-manager] elasticsearchmonitoring - New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearchMonitoring", :hosts=>["http://elasticsearch:9200"]} logstash | [WARN] 2020-08-04 15:17:49.714 [[.monitoring-logstash]-pipeline-manager] javapipeline - 'pipeline.ordered' is enabled and is likely less efficient, consider disabling if preserving event order is not necessary logstash | [INFO] 2020-08-04 15:17:49.718 [[.monitoring-logstash]-pipeline-manager] javapipeline - Starting pipeline {:pipeline_id=>".monitoring-logstash", "pipeline.workers"=>1, "pipeline.batch.size"=>2, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>2, "pipeline.sources"=>["monitoring pipeline"], :thread=>"#"} logstash | [INFO] 2020-08-04 15:17:49.793 [[.monitoring-logstash]-pipeline-manager] javapipeline - Pipeline started {"pipeline.id"=>".monitoring-logstash"} logstash | [INFO] 2020-08-04 15:17:49.802 [monitoring-license-manager] agent - Pipelines running {:count=>2, :running_pipelines=>[:main, :".monitoring-logstash"], :non_running_pipelines=>[]} elasticsearch | {"type": "server", "timestamp": "2020-08-04T15:19:18,117Z", "level": "INFO", "component": "o.e.m.j.JvmGcMonitorService", "cluster.name": "docker-cluster", "node.name": "09adc4351aef", "message": "[gc][145] overhead, spent [406ms] collecting in the last [1.5s]", "cluster.uuid": "ZvE8AxggQEmfWocw79CgCQ", "node.id": "2D0jukeSSM28Y_0wCpOyCg" } ``` ---
**Author:** ![kl29](https://avatars.discourse-cdn.com/v4/letter/k/c57346/32.png) [@kl29](https://discuss.elastic.co/u/kl29)\ **Post date:** [August 4, 2020, 3:32pm UTC](https://discuss.elastic.co/t/dockerizing-a-working-elk-project-dead-es-instance/243639/5 "2020-08-04T15:32:55Z")
```auto logstash | [INFO] 2020-08-04 15:16:48.017 [LogStash::Runner] licensereader - Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://elasticsearch:9200/]}} logstash | [WARN] 2020-08-04 15:16:48.528 [LogStash::Runner] licensereader - Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://elasticsearch:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://elasticsearch:9200/][Manticore::SocketException] Connection refused (Connection refused)"} logstash | [WARN] 2020-08-04 15:16:48.541 [LogStash::Runner] licensereader - Marking url as dead. Last error: [LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError] Elasticsearch Unreachable: [http://elasticsearch:9200/][Manticore::SocketException] Connection refused (Connection refused) {:url=>http://elasticsearch:9200/, :error_message=>"Elasticsearch Unreachable: [http://elasticsearch:9200/][Manticore::SocketException] Connection refused (Connection refused)", :error_class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError"} logstash | [ERROR] 2020-08-04 15:16:48.543 [LogStash::Runner] licensereader - Unable to retrieve license information from license server {:message=>"Elasticsearch Unreachable: [http://elasticsearch:9200/][Manticore::SocketException] Connection refused (Connection refused)"} ``` Here is where logstash's connection to a dead ES instance failed ---
**Author:** ![mossegarfw](https://avatars.discourse-cdn.com/v4/letter/m/71e660/32.png) [@mossegarfw](https://discuss.elastic.co/u/mossegarfw)\ **Post date:** [August 5, 2020, 10:42am UTC](https://discuss.elastic.co/t/dockerizing-a-working-elk-project-dead-es-instance/243639/6 "2020-08-05T10:42:41Z")
Thanks for sharing useful information with us.. It really helpful to [me](https://www.telldunkin.me/)..I always prefer to read the quality content and this thing I found in you post. thanks for sharing with us.. ---
**Author:** ![kl29](https://avatars.discourse-cdn.com/v4/letter/k/c57346/32.png) [@kl29](https://discuss.elastic.co/u/kl29)\ **Post date:** [August 9, 2020, 10:18pm UTC](https://discuss.elastic.co/t/dockerizing-a-working-elk-project-dead-es-instance/243639/7 "2020-08-09T22:18:09Z")
Hello once again! I am still stuck on this issue unfortunately. Any tips on why the Dockerized version cannot connect Logstash to ES, but the non-docker version is fine? Thanks! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 6, 2020, 10:18pm UTC](https://discuss.elastic.co/t/dockerizing-a-working-elk-project-dead-es-instance/243639/8 "2020-09-06T22:18:18Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.