Hi
I have deployed ELK in a Docker swarm.
Elasticsearch and kibana are in the same node, when logstash is in another one.
The issue is as follows, logstash is not able to connect to elasticsearch using the docker host, this is what I see in the logs:
....
[2019-09-24T12:39:18,516][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=>"Elasticsearch Unreachable: [http://:9200/][Manticore::ClientProtocolException] URI does not specify a valid host name: http:/_xpack"}
[2019-09-24T12:39:18,578][ERROR][logstash.monitoring.internalpipelinesource] Failed to fetch X-Pack information from Elasticsearch. This is likely due to failure to reach a live Elasticsearch cluster.
[2019-09-24T12:39:21,060][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"Java::JavaNet::URISyntaxException", :message=>"Illegal character in scheme name at index 8: outflink_elasticsearch:9200", :backtrace=>["java.net.URI$Parser.fail(java/net/URI.java:2848)", "java.net.URI$Parser.checkChars(java/net/URI.java:3021)", "java.net.URI$Parser.parse(java/net/URI.java:3048)", "java.net.URI.(java/net/URI.java:588)", "java.lang.reflect.Constructor.newInstance(java/lang/reflect/Constructor.java:423)", "org.jruby.javasupport.JavaConstructor.newInstanceDirect(org/jruby/javasupport/JavaConstructor.java:246)", "org.jruby.RubyClass.newInstance(org/jruby/RubyClass.java:1022)", "org.jruby.RubyClass$INVOKER$i$newInstance.call(org/jruby/RubyClass$INVOKER$i$newInstance.gen)",
(...)
[2019-09-24T12:39:21,142][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<LogStash::Error: Don't know how to handle `Java::JavaNet::URISyntaxException` for `PipelineAction::Create`>, :backtrace=>["org/logstash/execution/ConvergeResultExt.java:103:in `create'", "org/logstash/execution/ConvergeResultExt.java:34:in `add'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:329:in `block in converge_state'"]}
[2019-09-24T12:39:21,221][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit
This is my logstash.conf:
input {
tcp {
port => 5959
codec => json
}
}
filter {
grok {
match => {"message" => [
"%{DATESTAMP:date} %{WORD:event_type} %{GREEDYDATA:msg}",
"%{DATESTAMP:date} %{GREEDYDATA:msg}",
"%{GREEDYDATA:msg}"
]}
}
}
output {
elasticsearch {
hosts => ["outflink_elasticsearch:9200"]
index => "outflink_logs-%{+YYYY.MM.dd}"
}
}
BUT if I use the public domain instead of the docker host it works.
The problem is that I want to block access to elasticsearch from outside by avoiding opening the port 9200 from docker-compose, and use the container host instead from logstash and kibana.
What am I doing wrong here?
I was using versions 6.2.0, upgraded to 7.3.2 now and the problem persists, looks like something weird with the underscore in hosts.
Thanks in advance!