Logstash elasticsearch output plugin hangs at health check

I am using a Logstash ver 5.2 docker image and outputting to Elastic Cloud with ver 5.1.2. The appropriate environment variables have been set to correspond with my output.

output {
  stdout { codec => rubydebug }

  elasticsearch {
    hosts => ["${ELASTICSEARCH_HOST}"]
    user => "${ELASTIC_CLOUD_USER}"
    password => "${ELASTIC_CLOUD_PASSWORD}"
    index => "%{level}-%{+YYYY.MM.dd}"
    document_type => "logs"

If I comment out the elasticsearch output, then I am able to see stdout. Stdout indicates that my filters and input are coming in fine. However, if I leave the elasticsearch plugin enabled, then I get the following logs:

 Sending Logstash's logs to /var/log/logstash which is now configured via log4j2.properties
 19:21:31.746 [main] INFO  logstash.setting.writabledirectory - Creating directory {:setting=>"path.queue", :path=>"/usr/share/logstash/data/queue"}
 19:21:31.796 [LogStash::Runner] INFO  logstash.agent - No persistent UUID file found. Generating new UUID {:uuid=>"3715cecd-0db2-4a64-ac9d-c0c479fb085c", :path=>"/var/lib/logstash/uuid"}
 19:21:36.700 [[main]-pipeline-manager] INFO  logstash.outputs.elasticsearch - Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[https://username:password@b40c2d46d42bf8d09ffce40c191409fc.us-east-1.aws.found.io:9243/]}}
 19:21:36.719 [[main]-pipeline-manager] INFO  logstash.outputs.elasticsearch - Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>https://username:password@b40c2d46d42bf8d09ffce40c191409fc.us-east-1.aws.found.io:9243/, :path=>"/"}

On some days, I will actually see stdout with elasticsearch plugin enabled, but on other days without any change, I will get no other output from the "Running health check" line.

Edit: Removed my filters temporarily to confirm that it wasn't anything going on the filters end.

Also want to point out that these two users are having similar issue (not being able to send data from logstash to elastic)

Edit: To clarify, I do not have the password error like that mentioned in Pipeline aborted due to error in password field logstash

I was able to fix my issue by creating a logstash.yml with the following configuration:

  name: ${HOSTNAME}
    enabled: false
      username: ${ELASTIC_CLOUD_USER}
      password: ${ELASTIC_CLOUD_PASSWORD}

I also upgraded to use docker.elastic.co logstash 5.2.2 image.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.