I have created a pipeline to deduplicate logs in an index. I am facing issues while having logstash connect to the elasticsearch node.
IP hosting elasticsearch is 192.168.0.2 and has a self-signed certificate.
Following is the error, which I reckon is due to the self-signed certificate. How do solve it? (make it ignore the certificate authority not being in the root store.)
While I reckon it has no impact, logstash is running on a Raspebrry Pi running Raspbian OS (debian buster.)
Jun 16 01:01:33 HOSTNAME logstash[4912]: /usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:346:in `inputworker'
Jun 16 01:01:33 HOSTNAME logstash[4912]: /usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:337:in `block in start_input'
Jun 16 01:01:34 HOSTNAME logstash[4912]: ....[2020-06-16T01:01:34,263][ERROR][logstash.javapipeline ][main][3ac418918ce70506b0ac53d00cf562268311b9742cc89d6f145f89775a16f11a] A plugin had an unrecoverable error. Will restart this plugin.
Jun 16 01:01:34 HOSTNAME logstash[4912]: Pipeline_id:main
Jun 16 01:01:34 HOSTNAME logstash[4912]: Plugin: <LogStash::Inputs::Elasticsearch password=><password>, hosts=>["192.168.0.2:9200"], query=>"{ \"sort\": [ \"_doc\" ] }", index=>"cloned-logstash-2020.01.07", id=>"3ac418918ce70506b0ac53d00cf562268311b9742cc89d6f145f89775a16f11a", ssl=>true, user=>"elastic", enable_metric=>true, codec=><LogStash::Codecs::JSON id=>"json_4365356d-45ad-4f71-a3ed-d2a0efb03ca0", enable_metric=>true, charset=>"UTF-8">, size=>1000, scroll=>"1m", docinfo=>false, docinfo_target=>"@metadata", docinfo_fields=>["_index", "_type", "_id"]>
Jun 16 01:01:34 HOSTNAME logstash[4912]: Error: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Jun 16 01:01:34 HOSTNAME logstash[4912]: Exception: Manticore::ClientProtocolException
Jun 16 01:01:34 HOSTNAME logstash[4912]: Stack: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:37:in `block in initialize'
Jun 16 01:01:34 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:79:in `call'
Jun 16 01:01:34 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:274:in `call_once'
Jun 16 01:01:34 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:158:in `code'
Jun 16 01:01:34 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/http/manticore.rb:84:in `block in perform_request'
Jun 16 01:01:34 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/base.rb:262:in `perform_request'
Jun 16 01:01:34 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/http/manticore.rb:67:in `perform_request'
Jun 16 01:01:34 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/client.rb:131:in `perform_request'
Jun 16 01:01:34 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/elasticsearch-api-5.0.5/lib/elasticsearch/api/actions/search.rb:183:in `search'
Jun 16 01:01:34 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.6.0/lib/logstash/inputs/elasticsearch.rb:321:in `search_request'
Jun 16 01:01:34 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.6.0/lib/logstash/inputs/elasticsearch.rb:269:in `do_run_slice'
Jun 16 01:01:34 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.6.0/lib/logstash/inputs/elasticsearch.rb:250:in `do_run'
Jun 16 01:01:34 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.6.0/lib/logstash/inputs/elasticsearch.rb:238:in `run'
Jun 16 01:01:34 HOSTNAME logstash[4912]: /usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:346:in `inputworker'
Jun 16 01:01:34 HOSTNAME logstash[4912]: /usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:337:in `block in start_input'
Jun 16 01:01:35 HOSTNAME logstash[4912]: .[2020-06-16T01:01:35,359][ERROR][logstash.javapipeline ][main][3ac418918ce70506b0ac53d00cf562268311b9742cc89d6f145f89775a16f11a] A plugin had an unrecoverable error. Will restart this plugin.
Jun 16 01:01:35 HOSTNAME logstash[4912]: Pipeline_id:main
Jun 16 01:01:35 HOSTNAME logstash[4912]: Plugin: <LogStash::Inputs::Elasticsearch password=><password>, hosts=>["192.168.0.2:9200"], query=>"{ \"sort\": [ \"_doc\" ] }", index=>"cloned-logstash-2020.01.07", id=>"3ac418918ce70506b0ac53d00cf562268311b9742cc89d6f145f89775a16f11a", ssl=>true, user=>"elastic", enable_metric=>true, codec=><LogStash::Codecs::JSON id=>"json_4365356d-45ad-4f71-a3ed-d2a0efb03ca0", enable_metric=>true, charset=>"UTF-8">, size=>1000, scroll=>"1m", docinfo=>false, docinfo_target=>"@metadata", docinfo_fields=>["_index", "_type", "_id"]>
Jun 16 01:01:35 HOSTNAME logstash[4912]: Error: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Jun 16 01:01:35 HOSTNAME logstash[4912]: Exception: Manticore::ClientProtocolException
Jun 16 01:01:35 HOSTNAME logstash[4912]: Stack: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:37:in `block in initialize'
Jun 16 01:01:35 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:79:in `call'
Jun 16 01:01:35 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:274:in `call_once'
Jun 16 01:01:35 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:158:in `code'
Jun 16 01:01:35 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/http/manticore.rb:84:in `block in perform_request'
Jun 16 01:01:35 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/base.rb:262:in `perform_request'
Jun 16 01:01:35 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/http/manticore.rb:67:in `perform_request'
Jun 16 01:01:35 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/client.rb:131:in `perform_request'
Jun 16 01:01:35 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/elasticsearch-api-5.0.5/lib/elasticsearch/api/actions/search.rb:183:in `search'
Jun 16 01:01:35 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.6.0/lib/logstash/inputs/elasticsearch.rb:321:in `search_request'
Jun 16 01:01:35 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.6.0/lib/logstash/inputs/elasticsearch.rb:269:in `do_run_slice'
Jun 16 01:01:35 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.6.0/lib/logstash/inputs/elasticsearch.rb:250:in `do_run'
Jun 16 01:01:35 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.6.0/lib/logstash/inputs/elasticsearch.rb:238:in `run'
Jun 16 01:01:35 HOSTNAME logstash[4912]: /usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:346:in `inputworker'
Jun 16 01:01:35 HOSTNAME logstash[4912]: /usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:337:in `block in start_input'
Jun 16 01:01:36 HOSTNAME logstash[4912]: [2020-06-16T01:01:36,471][ERROR][logstash.javapipeline ][main][3ac418918ce70506b0ac53d00cf562268311b9742cc89d6f145f89775a16f11a] A plugin had an unrecoverable error. Will restart this plugin.
Jun 16 01:01:36 HOSTNAME logstash[4912]: Pipeline_id:main
Jun 16 01:01:36 HOSTNAME logstash[4912]: Plugin: <LogStash::Inputs::Elasticsearch password=><password>, hosts=>["192.168.0.2:9200"], query=>"{ \"sort\": [ \"_doc\" ] }", index=>"cloned-logstash-2020.01.07", id=>"3ac418918ce70506b0ac53d00cf562268311b9742cc89d6f145f89775a16f11a", ssl=>true, user=>"elastic", enable_metric=>true, codec=><LogStash::Codecs::JSON id=>"json_4365356d-45ad-4f71-a3ed-d2a0efb03ca0", enable_metric=>true, charset=>"UTF-8">, size=>1000, scroll=>"1m", docinfo=>false, docinfo_target=>"@metadata", docinfo_fields=>["_index", "_type", "_id"]>
Jun 16 01:01:36 HOSTNAME logstash[4912]: Error: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Jun 16 01:01:36 HOSTNAME logstash[4912]: Exception: Manticore::ClientProtocolException
Jun 16 01:01:36 HOSTNAME logstash[4912]: Stack: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:37:in `block in initialize'
Jun 16 01:01:36 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:79:in `call'
Jun 16 01:01:36 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:274:in `call_once'
Jun 16 01:01:36 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:158:in `code'
Jun 16 01:01:36 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/http/manticore.rb:84:in `block in perform_request'
Jun 16 01:01:36 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/base.rb:262:in `perform_request'
Jun 16 01:01:36 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/http/manticore.rb:67:in `perform_request'
Jun 16 01:01:36 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/client.rb:131:in `perform_request'
Jun 16 01:01:36 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/elasticsearch-api-5.0.5/lib/elasticsearch/api/actions/search.rb:183:in `search'
Jun 16 01:01:36 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.6.0/lib/logstash/inputs/elasticsearch.rb:321:in `search_request'
Jun 16 01:01:36 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.6.0/lib/logstash/inputs/elasticsearch.rb:269:in `do_run_slice'
Jun 16 01:01:36 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.6.0/lib/logstash/inputs/elasticsearch.rb:250:in `do_run'
Jun 16 01:01:36 HOSTNAME logstash[4912]: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.6.0/lib/logstash/inputs/elasticsearch.rb:238:in `run'