Logstash not able to connect to elasticsearch docker host

Elastic Stack Logstash
1

Hi
I have deployed ELK in a Docker swarm.
Elasticsearch and kibana are in the same node, when logstash is in another one.

The issue is as follows, logstash is not able to connect to elasticsearch using the docker host, this is what I see in the logs:

....
[2019-09-24T12:39:18,516][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=>"Elasticsearch Unreachable: [http://:9200/][Manticore::ClientProtocolException] URI does not specify a valid host name: http:/_xpack"}
[2019-09-24T12:39:18,578][ERROR][logstash.monitoring.internalpipelinesource] Failed to fetch X-Pack information from Elasticsearch. This is likely due to failure to reach a live Elasticsearch cluster.
[2019-09-24T12:39:21,060][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"Java::JavaNet::URISyntaxException", :message=>"Illegal character in scheme name at index 8: outflink_elasticsearch:9200", :backtrace=>["java.net.URI$Parser.fail(java/net/URI.java:2848)", "java.net.URI$Parser.checkChars(java/net/URI.java:3021)", "java.net.URI$Parser.parse(java/net/URI.java:3048)", "java.net.URI.(java/net/URI.java:588)", "java.lang.reflect.Constructor.newInstance(java/lang/reflect/Constructor.java:423)", "org.jruby.javasupport.JavaConstructor.newInstanceDirect(org/jruby/javasupport/JavaConstructor.java:246)", "org.jruby.RubyClass.newInstance(org/jruby/RubyClass.java:1022)", "org.jruby.RubyClass$INVOKER$i$newInstance.call(org/jruby/RubyClass$INVOKER$i$newInstance.gen)",
(...)
[2019-09-24T12:39:21,142][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<LogStash::Error: Don't know how to handle `Java::JavaNet::URISyntaxException` for `PipelineAction::Create`>, :backtrace=>["org/logstash/execution/ConvergeResultExt.java:103:in `create'", "org/logstash/execution/ConvergeResultExt.java:34:in `add'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:329:in `block in converge_state'"]}
[2019-09-24T12:39:21,221][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

This is my logstash.conf:

input {
  tcp {
    port => 5959
    codec => json
  }
}

filter {
    grok {
        match => {"message" => [
        "%{DATESTAMP:date} %{WORD:event_type} %{GREEDYDATA:msg}",
        "%{DATESTAMP:date} %{GREEDYDATA:msg}",
                "%{GREEDYDATA:msg}"
        ]}
    }
}

output {
    elasticsearch {
      hosts => ["outflink_elasticsearch:9200"]
      index => "outflink_logs-%{+YYYY.MM.dd}"
    }
}

BUT if I use the public domain instead of the docker host it works.
The problem is that I want to block access to elasticsearch from outside by avoiding opening the port 9200 from docker-compose, and use the container host instead from logstash and kibana.

What am I doing wrong here?
I was using versions 6.2.0, upgraded to 7.3.2 now and the problem persists, looks like something weird with the underscore in hosts.

Thanks in advance!

2

Hello, remove _ in your hostname, it solved the problem for me with logstash 7.3.2

3

Not that easy, I'm using docker deploy and it's automatically adding the _ to the hosts names.

4

i dont use docker deploy but if you use a docker-compose file you can force the hostname of your container using hostname:

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.