[logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance - AWS Govcloud

Getting the following error when attempting to send on-prem logstash to elasticsearch in the cloud

The version running in the cloud is 8.2.2

Log file

 [2022-11-22T08:24:43,581][INFO ][logstash.runner          ] Log4j configuration path used is: C:\logstash\config\log4j2.properties 
[2022-11-22T08:24:43,596][WARN ][deprecation.logstash.runner] The use of JAVA_HOME has been deprecated. Logstash 8.0 and later ignores JAVA_HOME and uses the bundled JDK. Running Logstash with the bundled JDK is recommended. The bundled JDK has been verified to work with each specific version of Logstash, and generally provides best performance and reliability. If you have compelling reasons for using your own JDK (organizational-specific compliance requirements, for example), you can configure LS_JAVA_HOME to use that version instead.
[2022-11-22T08:24:43,596][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.16.2", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.6+10 on 11.0.6+10 +indy +jit [mswin32-x86_64]"}
[2022-11-22T08:24:48,643][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[2022-11-22T08:24:49,487][INFO ][org.reflections.Reflections] Reflections took 94 ms to scan 1 urls, producing 119 keys and 417 values 
[2022-11-22T08:24:51,628][WARN ][deprecation.logstash.codecs.plain] Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode.
[2022-11-22T08:24:51,753][WARN ][deprecation.logstash.inputs.jdbc] Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode.
[2022-11-22T08:24:51,909][WARN ][deprecation.logstash.codecs.plain] Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode.
[2022-11-22T08:24:52,034][WARN ][deprecation.logstash.outputs.elasticsearch] Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode.
[2022-11-22T08:24:53,128][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//127.0.0.1"]}
[2022-11-22T08:24:53,472][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[https://elastic:xxxxxx@xxxxxxxxxxxxxxxxxxxxxxxxxx.us-gov-east-1.aws.elastic-cloud.com:443/]}}
[2022-11-22T08:24:53,862][WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error {:url=>"https://elastic:xxxxxx@xxxxxxxxxxxxxxxxxxxxxxxxxx.us-gov-east-1.aws.elastic-cloud.com:443/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :message=>"Elasticsearch Unreachable: [https://elastic:xxxxxx@xxxxxxxxxxxxxxxxxxxxxxxxxx.us-gov-east-1.aws.elastic-cloud.com:443/][Manticore::ClientProtocolException] PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"}
[2022-11-22T08:24:53,909][WARN ][logstash.javapipeline    ] 'pipeline.ordered' is enabled and is likely less efficient, consider disabling if preserving event order is not necessary
[2022-11-22T08:24:53,956][INFO ][logstash.javapipeline    ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>125, "pipeline.sources"=>["C:/logstash/config/main.conf"], :thread=>"#<Thread:0x7e77447d run>"}
[2022-11-22T08:24:54,519][INFO ][logstash.javapipeline    ] Pipeline Java execution initialization time {"seconds"=>0.55}
[2022-11-22T08:24:54,878][INFO ][logstash.javapipeline    ] Pipeline started {"pipeline.id"=>"main"}
[2022-11-22T08:24:54,987][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2022-11-22T08:24:58,972][WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error {:url=>"https://elastic:xxxxxx@xxxxxxxxxxxxxxxxxxxxxxxxxx.us-gov-east-1.aws.elastic-cloud.com:443/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :message=>"Elasticsearch Unreachable: [https://elastic:xxxxxx@xxxxxxxxxxxxxxxxxxxxxxxxxx.us-gov-east-1.aws.elastic-cloud.com:443/][Manticore::ClientProtocolException] PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"}
[2022-11-22T08:25:04,097][WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error {:url=>"https://elastic:xxxxxx@xxxxxxxxxxxxxxxxxxxxxxxxxx.us-gov-east-1.aws.elastic-cloud.com:443/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :message=>"Elasticsearch Unreachable: [https://elastic:xxxxxx@xxxxxxxxxxxxxxxxxxxxxxxxxx.us-gov-east-1.aws.elastic-cloud.com:443/][Manticore::ClientProtocolException] PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"} 

Conf output

output {
  elasticsearch {
    cloud_id => "cloud_id"
    cloud_auth => "elastic:xxxxxxxx"
    index => "mssql"
  }
}

Have you set HTTPS mode on Elasticsearch? You need at least SSL mode enabled without CA validation.

ssl => true
ssl_certificate_verification => false

After that you have to set the certificate.

ssl => true
ssl_certificate_verification => true
cacert => "/path/to/http_ca.crt"

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.