Fresh instalation - Logstash cannot connect to ElasticSearch

Elastic Stack Logstash
1

Hi all,

I installed Elasticsearch, Kibana, and Logstash from zip files on Windows.
Elasticsearch seems to work - I can connect via Kibana and insert documents to the index. My browser returns JSON response when directed to localhost:9200
Everything installed on the local machine on default settings - no changes to the configuration.
But Logstash cannot connect. I have run logstash-sample.conf using following command:
c:\logstash-7.3.2\bin\logstash.bat --path.data c:\logstash-7.3.2\data\ -f c:\logstash-7.3.2\config\logstash-sample.conf
and I see following in the output:

[2019-09-18T15:57:00,323][INFO ][logstash.outputs.elasticsearch] Installing elasticsearch template to _template/logstash
[2019-09-18T15:57:00,358][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>}
[2019-09-18T15:57:00,382][INFO ][org.logstash.beats.Server] Starting server on port: 5044
[2019-09-18T15:57:00,831][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2019-09-18T15:58:00,410][WARN ][logstash.outputs.elasticsearch] Marking url as dead. Last error: [LogStash::Outputs::Elasticsearch::HttpClient::Pool::HostUnreachableError] Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketTimeout] Read timed out {:url=>http://localhost:9200/, :error_message=>"Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketTimeout] Read timed out", :error_class=>"LogStash::Outputs::Elasticsearch::HttpClient::Pool::HostUnreachableError"}
[2019-09-18T15:58:00,414][ERROR][logstash.outputs.elasticsearch] Failed to install template. {:message=>"Elasticsearch Unreachable: [http://localhost:9200/][Manticore::SocketTimeout] Read timed out", :class=>"LogStash::Outputs::Elasticsearch::HttpClient::Pool::HostUnreachableError", :backtrace=>["C:/logstash-7.3.2/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:293:in `perform_request_to_url'", "C:/logstash-7.3.2/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:278:in `block in perform_request'", "C:/logstash-7.3.2/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:373:in `with_connection'", "C:/logstash-7.3.2/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:277:in `perform_request'", "C:/logstash-7.3.2/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:285:in `block in Pool'", "C:/logstash-7.3.2/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client.rb:352:in `template_put'", "C:/logstash-7.3.2/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/http_client.rb:86:in `template_install'", "C:/logstash-7.3.2/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/template_manager.rb:28:in `install'", "C:/logstash-7.3.2/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/template_manager.rb:16:in `install_template'", "C:/logstash-7.3.2/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/common.rb:130:in `install_template'", "C:/logstash-7.3.2/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.1.0-java/lib/logstash/outputs/elasticsearch/common.rb:51:in `block in setup_after_successful_connection'"]}

What am I missing or doing wrong?

2

It is connecting, but not getting a response when it tries to install a template. How is the elasticsearch output configured?

3

I do not know if I understand your question corectly. Here is output section of sample conf file:

output {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
#user => "elastic"
#password => "changeme"
}
}

I tried with user and password parameters uncommented, but still got same error.

4

By default, Logstash tries to install a template for an Index within ES for every request it sends. The error indicates that the user "elastic" used in the output section of the Logstash's pipeline does not have enough privileges to create it.

So provide all the privileges via the Security to start and then refine it as per your needs.

5

Can it be done with basic features? All documentation I can find states, that one need X-Pack to perform such actions.

6

The Security feature is part of XPack Basic license. And XPack is part of v7.3.2 by default. It just needs to be enabled in the config.

Some documentation on this topic:

7

Thank you, this was a case.
I followed documentation from links you provided and my stack works perfectly now.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.