# Logstash docker image keeps resetting elasticsearch URL

**URL:** <https://discuss.elastic.co/t/logstash-docker-image-keeps-resetting-elasticsearch-url/99229>\
**Category:** Logstash\
**Created:** [September 3, 2017, 6:54pm UTC](https://discuss.elastic.co/t/logstash-docker-image-keeps-resetting-elasticsearch-url/99229 "2017-09-03T18:54:51Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![sfchrisgleason](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sfchrisgleason/32/21782_2.png) [@sfchrisgleason](https://discuss.elastic.co/u/sfchrisgleason)\
**Post date:** [September 3, 2017, 6:54pm UTC](https://discuss.elastic.co/t/logstash-docker-image-keeps-resetting-elasticsearch-url/99229/1 "2017-09-03T18:54:51Z")

</div>

Hello all,

I am experimenting with bringing up an ELK stack across multiple physical machines using the docker images (Docker running independently on each machine). I'm using the official images from elastic using:

```
docker pull logstash
docker pull elasticsearch
docker pull kibana

```

I've gotten all three up, but whenever I try to ingest via beats, it's telling me in Kibana that there isn't any data in the filebeat-\* index that matches the pattern.

On Logstash I keep seeing these lines when restarting:

> 18:30:42.309 [Ruby-0-Thread-8: /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-7.3.8-java/lib/logstash/outputs/elasticsearch/http\_client/pool.rb:136] INFO logstash.outputs.elasticsearch - Elasticsearch pool URLs updated {:changes=\>{:removed=\>[[http://172.x.x.x:9200/](http://172.x.x.x:9200/)], :added=\>[[http://192](http://192)..x.x.x:9200/]}}

The 172 address is what I've got specified in the es-output for logstash. I've also added that as network.host in elasticsearch.yml inside the ES docker container.

I don't have [xpack installed](https://discuss.elastic.co/t/elasticsearch-unreachable-error-in-logstash/75157/8) at least I think, and I've [set sniffing to false](https://discuss.elastic.co/t/logstash-elasticsearch-output-plugin-issue-unexpected-pool-error/64899/7).

I haven't been able to get the url to stick.

Any ideas? Thanks ahead of time!

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [September 4, 2017, 5:29am UTC](https://discuss.elastic.co/t/logstash-docker-image-keeps-resetting-elasticsearch-url/99229/2 "2017-09-04T05:29:38Z")

</div>

> I’m using the official images from elastic using:
> 
> ```
> docker pull logstash
> docker pull elasticsearch
> docker pull kibana
> 
> ```

Unless you've configured your Docker Engine to use [docker.elastic.co](http://docker.elastic.co) as your upstream registry you're not using the official Elastic images.

Are you restarting the containers at some point? How are you updating the configuration files?

---

<div class="post-metadata">

**Author:** ![sfchrisgleason](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sfchrisgleason/32/21782_2.png) [@sfchrisgleason](https://discuss.elastic.co/u/sfchrisgleason)\
**Post date:** [September 4, 2017, 10:04am UTC](https://discuss.elastic.co/t/logstash-docker-image-keeps-resetting-elasticsearch-url/99229/3 "2017-09-04T10:04:50Z")

</div>

Yeah sorry my eyes saw elastic in there somewhere, but it does look like they are [docker.io](http://docker.io) ones

[docker.io/elasticsearch](http://docker.io/elasticsearch) latest 7516701e4922 13 days ago 315.5 MB

I am restarting the containers yes. That's when I'm seeing the es url replacement in the logs. After logstash says it has replaced the ES URL I'm seeing these messages:

> 05:16:16.713 [Ruby-0-Thread-8: /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-7.3.8-java/lib/logstash/outputs/elasticsearch/http\_client/pool.rb:136] WARN logstash.outputs.elasticsearch - Elasticsearch output attempted to sniff for new connections but cannot. No living connections are detected. Pool contains the following current URLs {:url\_info=\>{[http://192.168.1.1:9200/=](http://192.168.1.1:9200/=)\>{:in\_use=\>0, :state=\>:dead, :last\_error=\>#\<LogStash::Outputs::Elasticsearch::HttpClient::Pool::HostUnreachableError: Could not reach host Manticore::SocketException: Connection refused (Connection refused)\>, :last\_errored\_at=\>2017-09-03 18:30:47 +0000}}}  
> 05:16:17.477 [Ruby-0-Thread-7: /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-7.3.8-java/lib/logstash/outputs/elasticsearch/http\_client/pool.rb:224] INFO logstash.outputs.elasticsearch - Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[http://192.1.x.x:9200/](http://192.1.x.x:9200/), :path=\>"/"}

Here is my Elasticsearh output

```
output {
  elasticsearch {
    hosts => ["172.24.x.x:9200"]
    sniffing => false
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }
}

```

Thanks!

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [September 4, 2017, 11:11am UTC](https://discuss.elastic.co/t/logstash-docker-image-keeps-resetting-elasticsearch-url/99229/4 "2017-09-04T11:11:45Z")

</div>

Okay, but how are you updating the configuration files? Are they residing in a host-mounted directory or an otherwise persistent volume? How are you starting the containers?

---

<div class="post-metadata">

**Author:** ![sfchrisgleason](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sfchrisgleason/32/21782_2.png) [@sfchrisgleason](https://discuss.elastic.co/u/sfchrisgleason)\
**Post date:** [September 4, 2017, 11:54am UTC](https://discuss.elastic.co/t/logstash-docker-image-keeps-resetting-elasticsearch-url/99229/5 "2017-09-04T11:54:58Z")

</div>

Using docker exec -it container bash

then manually updating, then commiting the image so I can restart as necessary.

here are the commands I'm using for all three products currently (though have tried many different variations):

```
docker run --name elasticsearch -v "/opt/elasticsearch/esdata":/usr/share/elasticsearch/data -e network.host=172.24.x.x -p 9200:9200 -p 9300:9300 elasticsearch

sudo docker run -d --name kibana -p 5601:5601 -e ELASTICSEARCH_URL=http://172.24.x.x:9200 --net host kibana

sudo docker run -d --name="logstash" -v /etc/logstash/conf.d:/etc/logstash/conf.d:ro -p 5044:5044 -v /var/log/logstash:/host/var/log --net bridge logstash -f /etc/logstash/conf.d --debug

```

Thanks!

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [September 4, 2017, 6:06pm UTC](https://discuss.elastic.co/t/logstash-docker-image-keeps-resetting-elasticsearch-url/99229/6 "2017-09-04T18:06:03Z")

</div>

Um, but you have a read-only host mount of /etc/logstash/conf.d. Are you actually able to modify files there? Are those changes visible in the host directory?

---

<div class="post-metadata">

**Author:** ![sfchrisgleason](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sfchrisgleason/32/21782_2.png) [@sfchrisgleason](https://discuss.elastic.co/u/sfchrisgleason)\
**Post date:** [September 4, 2017, 7:49pm UTC](https://discuss.elastic.co/t/logstash-docker-image-keeps-resetting-elasticsearch-url/99229/7 "2017-09-04T19:49:42Z")

</div>

I feel like a big dope, but I found the source of my problem (at least this one).

I accidentally created two elasticsearch outputs and one was superceding the other with a sniffing =\> true

The ES errors have all gone away in logstash now.

Now I just need to work out the networking issues inbound to filebeat!

Thanks Magus!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 2, 2017, 7:49pm UTC](https://discuss.elastic.co/t/logstash-docker-image-keeps-resetting-elasticsearch-url/99229/8 "2017-10-02T19:49:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.