Cant use logstash with elasticsearch URL

Elastic Stack Elasticsearch
1

I am using Elasticsearch and Logstash as Docker containers (within the same network) for my Django project.

Logstash fails to connect to Elasticsearch when using hosts => ["localhost:9200"] in logstash.conf (error: "failed: Connection refused").
However, it works if I use the IP assigned by Docker (docker network inspect), which is not ideal for a large project.

network.host: 0.0.0.0 in elasticsearch.yml (as recommended in many forum posts).
Both containers are running **within the same Docker network

Why can't Logstash connect to Elasticsearch via localhost? What is the correct way to configure this?

question
question850×870 81.6 KB

2

Because localhost means the same host, in this case, the same container, which seems to not be the case.

If you have a Logstash container and you configure the elasticsearch output of a pipeline to connect to localhost:9200, then logstash will try to connect to a Elasticsearch instance running on the same container.

If both Logstash and Elasticsearch are on the same docker network, then you need to access then by using the container name, for example, if your Elasticsearch container is named elasticsearch, then you need to use this name in the logstash output, hosts => ["http://elasticsearch:9200"].

Are you using compose? Can you share how you are starting up your containers.

1 Like
3

Hi, thanks for advise.

I figured out that setting xpack.security.enabled: false in elasticsearch.yml helps with this issue. It doesnt bother me, as I dont use authentication or smth like that, just logs. However I still dont understand how typing a certain IP is better, than some abstract elasticsearch url.

And as I know with Kibana this field has to be True. Luckily I use grafana.

1 Like
4

This has nothing to do with Logstash or Elasticsearch, or even Docker, it is how network works.

The hostname localhost is a special hostname that refers to the current computer or container.

When you use localhost in any configuration of any system it means that you want to access something that is running on the same computer/container that is making the request.

So, using localhost in your Logstash configuration means that you want to access a Elasticsearch instance running on the same container, but there is no Elasticsearch instance on the same container, it is running on a different container on the same network.

When you use docker and you have multiple containers on the same network, you can access then by using the container name, which is used by docker as the hostname for the service in the internal network, so if you Elasticsearch container is named elasticsearch, then you can use this in the configurations to access it.

You can use Kibana with authentication disabled, Kibana is a client for Elasticsearch, the authentication is done in Elasticsearch, if it is disabled, then Kibana can be used without authentication.

1 Like