Having trouble to get ELK working on the Logstash part

Elastic Stack Elasticsearch
1

I am trying out the ELK to pipe a tomcat log to ES via Logstash and view it in Kibana. After running "docker-compose up" I can visit ES via http://localhost:9200/_cat/health. Also Kibana via http://localhost:5601. However, Logstash always gives

[logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://localhost:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketException] Connection refused (Connection refused)"}

Here is my docker-compose.yml

version: '3'
services:
  elasticsearch:
    container_name: elasticsearch
    image: docker.elastic.co/elasticsearch/elasticsearch:7.4.1
    environment:
      - node.name=elasticsearch
      - cluster.name=elasticsearch
      - bootstrap.memory_lock=true
      - discovery.type=single-node
    ports:
      - "9200:9200"
    ulimits:
      memlock:
        soft: -1
        hard: -1
  kibana:
    container_name: kibana
    image: docker.elastic.co/kibana/kibana:7.4.1
    ports:
      - "5601:5601"
  logstash:
    container_name: logstash
    image: docker.elastic.co/logstash/logstash:7.4.1
    volumes:
      - ./logstash.conf:/usr/share/logstash/pipeline/logstash.conf

And my logstash.conf

input {
  file {
    path => "C:\Tomcat\logs\myapp.log"
    codec => "json"
    type => "logback"
  }
}
output {
  elasticsearch {
    hosts => "localhost:9200"
  }
}

Please help

2

In the logstash output section try below, elasticsearch is the name of the container within the docker context

hosts => "http://elasticsearch:9200"

Also if this is on Windows use forward slashes for the path to the log file.

path => "C:/Tomcat/logs/myapp.log"

1 Like
3

Thanks @stephenb! The error has gone once I changed "localhost" to "elasticsearch"!
As for the path, I have to mount a volume for "C:\Tomcat\logs\myapp.log" to "/usr/share/logstash/logs/myapp.log:ro" and update the input file path to "/usr/share/logstash/logs/myapp.log".

Now it all works! I wonder why localhost did not work though?

4

Hi 1 other thing you should probably create a network in your docker compose. Example here

And localhost for logstash is it's on container not the Elasticsearch container the others work because you bound the port from container to external host

1 Like
5

Very helpful @stephenb thank you!
I checked, the docker images are already creating and joining the same network called "elk_default" out of the box. So I guess thats enough :ok_hand:

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.