Logstash always tries to connect to ES

Elastic Stack Logstash
1

I am forwarding logs from Logstash to Kafka but Logstash always tries to connect to ES at http://localhost:9200/

[2017-05-18T03:45:49,875][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2017-05-18T03:45:50,383][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://logstash_system:xxxxxx@localhost:9200/, :path=>"/"}
[2017-05-18T03:45:50,401][WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>#<URI::HTTP:0x4ac80b74 URL:http://logstash_system:xxxxxx@localhost:9200/_xpack/monitoring/?system_id=logstash&system_api_version=2&interval=1s>, :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://logstash_system:xxxxxx@localhost:9200/][Manticore::SocketException] Connection refused (Connection refused)"}
[2017-05-18T03:45:55,403][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://logstash_system:xxxxxx@localhost:9200/, :path=>"/"}
[2017-05-18T03:45:55,425][WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>#<URI::HTTP:0x5ed9072f URL:http://logstash_system:xxxxxx@localhost:9200/_xpack/monitoring/?system_id=logstash&system_api_version=2&interval=1s>, :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://logstash_system:xxxxxx@localhost:9200/][Manticore::SocketException] Connection refused (Connection refused)"}
[2017-05-18T03:45:55,543][WARN ][logstash.outputs.elasticsearch] Marking url as dead. Last error: [LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError] Elasticsearch Unreachable: [http://logstash_system:xxxxxx@localhost:9200/_xpack/monitoring/?system_id=logstash&system_api_version=2&interval=1s][Manticore::SocketException] Connection refused (Connection refused) {:url=>http://logstash_system:xxxxxx@localhost:9200/_xpack/monitoring/?system_id=logstash&system_api_version=2&interval=1s, :error_message=>"Elasticsearch Unreachable: [http://logstash_system:xxxxxx@localhost:9200/_xpack/monitoring/?system_id=logstash&system_api_version=2&interval=1s][Manticore::SocketException] Connection refused (Connection refused)", :error_class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError"}
[2017-05-18T03:45:55,546][ERROR][logstash.outputs.elasticsearch] Attempted to send a bulk request to elasticsearch' but Elasticsearch appears to be unreachable or down! {:error_message=>"Elasticsearch Unreachable: [http://logstash_system:xxxxxx@localhost:9200/_xpack/monitoring/?system_id=logstash&system_api_version=2&interval=1s][Manticore::SocketException] Connection refused (Connection refused)", :class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError", :will_retry_in_seconds=>2}
[2017-05-18T03:45:57,550][WARN ][logstash.outputs.elasticsearch] UNEXPECTED POOL ERROR {:e=>#<LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError: No Available connections>}
[2017-05-18T03:45:57,556][ERROR][logstash.outputs.elasticsearch] Attempted to send a bulk request to elasticsearch, but no there are no living connections in the connection pool. Perhaps Elasticsearch is unreachable or down? {:error_message=>"No Available connections", :class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError", :will_retry_in_seconds=>4}

This is logstash.conf

input {
  beats {
    port => 5044
  }
}

output {
  kafka {
    ...
  }
}

Why does Logstash try to connect to ES and how to fix it?

2

Is it a bug?

3

If you are using x-pack then LS will try to send monitoring data to Elasticsearch. This is a feature. Without x-pack LS will use an inbuilt Webserver to allow for monitoring data querying.

1 Like
4

Thanks. It looks like it's not possible to disable X-Pack for Logstash:

General Monitoring Settings

xpack.monitoring.enabled
    Set to false to disable X-Pack monitoring. Configure in both elasticsearch.yml and kibana.yml
5

It looks like it's not possible to disable X-Pack for Logstash:

I'm confused. You just quoted the section in the documentation that tells you how to disable it.

6

I found that in Kibana documentation only. Works for Logstash as well.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.