Hi elastic team
I've configured a pipeline containing a file input, some filters and elasticsearch output. the target es server is secured by ssl and the certificates are generated using the the way explained in docs. note that the es and logstash versions are 6.8
the problem I facing is that when I use the elastic-stack-ca.p12 file for 'cacert' param in elasticsearch output, the following error is generated in logstash (signed fields invalid):
[2023-01-01T16:41:03,368][ERROR][logstash.pipeline ] Error registering plugin {:pipeline_id=>"main", :plugin=>"#<LogStash::OutputDelegator:0x24c5b7c5>", :error=>"signed fields invalid", :thread=>"#<Thread:0x6b3a135e run>"}
[2023-01-01T16:41:03,369][ERROR][logstash.pipeline ] Pipeline aborted due to error {:pipeline_id=>"main", :exception=>java.security.cert.CertificateParsingException: signed fields invalid, :backtrace=>["sun.security.x509.X509CertImpl.parse(sun/security/x509/X509CertImpl.java:1829)", "sun.security.x509.X509CertImpl.<init>(sun/security/x509/X509CertImpl.java:194)", "sun.security.provider.X509Factory.parseX509orPKCS7Cert(sun/security/provider/X509Factory.java:476)", "sun.security.provider.X509Factory.engineGenerateCertificates(sun/security/provider/X509Factory.java:361)", "java.security.cert.CertificateFactory.generateCertificates(java/security/cert/CertificateFactory.java:478)", "jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)", "jdk.internal.reflect.NativeMethodAccessorImpl.invoke(jdk/internal/reflect/NativeMethodAccessorImpl.java:62)", "jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(jdk/internal/reflect/DelegatingMethodAccessorImpl.java:43)", "java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:566)", "org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(org/jruby/javasupport/JavaMethod.java:455)", "org.jruby.javasupport.JavaMethod.invokeDirect(org/jruby/javasupport/JavaMethod.java:316)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.manticore_minus_0_dot_7_dot_0_minus_java.lib.manticore.client.setup_trust_store(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.7.0-java/lib/manticore/client.rb:635)", "org.jruby.RubyIO.ensureYieldClose(org/jruby/RubyIO.java:1163)", "org.jruby.RubyIO.open(org/jruby/RubyIO.java:1157)", "org.jruby.RubyKernel.open(org/jruby/RubyKernel.java:320)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.manticore_minus_0_dot_7_dot_0_minus_java.lib.manticore.client.setup_trust_store(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.7.0-java/lib/manticore/client.rb:634)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.manticore_minus_0_dot_7_dot_0_minus_java.lib.manticore.client.ssl_socket_factory_from_options(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.7.0-java/lib/manticore/client.rb:622)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.manticore_minus_0_dot_7_dot_0_minus_java.lib.manticore.client.pool_builder(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.7.0-java/lib/manticore/client.rb:398)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.manticore_minus_0_dot_7_dot_0_minus_java.lib.manticore.client.pool(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.7.0-java/lib/manticore/client.rb:406)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.manticore_minus_0_dot_7_dot_0_minus_java.lib.manticore.client.initialize(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.7.0-java/lib/manticore/client.rb:209)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_output_minus_elasticsearch_minus_9_dot_4_dot_0_minus_java.lib.logstash.outputs.elasticsearch.http_client.manticore_adapter.initialize(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-9.4.0-java/lib/logstash/outputs/elasticsearch/http_client/manticore_adapter.rb:26)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_output_minus_elasticsearch_minus_9_dot_4_dot_0_minus_java.lib.logstash.outputs.elasticsearch.http_client.build_adapter(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-9.4.0-java/lib/logstash/outputs/elasticsearch/http_client.rb:282)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_output_minus_elasticsearch_minus_9_dot_4_dot_0_minus_java.lib.logstash.outputs.elasticsearch.http_client.build_pool(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-9.4.0-java/lib/logstash/outputs/elasticsearch/http_client.rb:286)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_output_minus_elasticsearch_minus_9_dot_4_dot_0_minus_java.lib.logstash.outputs.elasticsearch.http_client.initialize(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-9.4.0-java/lib/logstash/outputs/elasticsearch/http_client.rb:64)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_output_minus_elasticsearch_minus_9_dot_4_dot_0_minus_java.lib.logstash.outputs.elasticsearch.http_client_builder.create_http_client(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-9.4.0-java/lib/logstash/outputs/elasticsearch/http_client_builder.rb:103)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_output_minus_elasticsearch_minus_9_dot_4_dot_0_minus_java.lib.logstash.outputs.elasticsearch.http_client_builder.build(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-9.4.0-java/lib/logstash/outputs/elasticsearch/http_client_builder.rb:99)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_output_minus_elasticsearch_minus_9_dot_4_dot_0_minus_java.lib.logstash.outputs.elasticsearch.build_client(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-9.4.0-java/lib/logstash/outputs/elasticsearch.rb:238)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_output_minus_elasticsearch_minus_9_dot_4_dot_0_minus_java.lib.logstash.outputs.elasticsearch.RUBY$method$build_client$0$__VARARGS__(usr/share/logstash/vendor/bundle/jruby/$2_dot_5_dot_0/gems/logstash_minus_output_minus_elasticsearch_minus_9_dot_4_dot_0_minus_java/lib/logstash/outputs//usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-9.4.0-java/lib/logstash/outputs/elasticsearch.rb)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_output_minus_elasticsearch_minus_9_dot_4_dot_0_minus_java.lib.logstash.outputs.elasticsearch.common.register(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-9.4.0-java/lib/logstash/outputs/elasticsearch/common.rb:25)", "org.jruby.RubyClass.finvoke(org/jruby/RubyClass.java:548)", "org.jruby.RubyBasicObject.callMethod(org/jruby/RubyBasicObject.java:354)", "org.logstash.config.ir.compiler.OutputStrategyExt$SimpleAbstractOutputStrategyExt.reg(org/logstash/config/ir/compiler/OutputStrategyExt.java:246)", "org.logstash.config.ir.compiler.OutputStrategyExt$AbstractOutputStrategyExt.register(org/logstash/config/ir/compiler/OutputStrategyExt.java:106)", "org.logstash.config.ir.compiler.OutputDelegatorExt.doRegister(org/logstash/config/ir/compiler/OutputDelegatorExt.java:91)", "org.logstash.config.ir.compiler.AbstractOutputDelegatorExt.register(org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:48)", "usr.share.logstash.logstash_minus_core.lib.logstash.pipeline.register_plugin(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:259)", "usr.share.logstash.logstash_minus_core.lib.logstash.pipeline.RUBY$method$register_plugin$0$__VARARGS__(usr/share/logstash/logstash_minus_core/lib/logstash//usr/share/logstash/logstash-core/lib/logstash/pipeline.rb)", "usr.share.logstash.logstash_minus_core.lib.logstash.pipeline.register_plugins(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:270)", "org.jruby.RubyArray.each(org/jruby/RubyArray.java:1792)", "usr.share.logstash.logstash_minus_core.lib.logstash.pipeline.register_plugins(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:270)", "usr.share.logstash.logstash_minus_core.lib.logstash.pipeline.RUBY$method$register_plugins$0$__VARARGS__(usr/share/logstash/logstash_minus_core/lib/logstash//usr/share/logstash/logstash-core/lib/logstash/pipeline.rb)", "usr.share.logstash.logstash_minus_core.lib.logstash.pipeline.maybe_setup_out_plugins(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:611)", "usr.share.logstash.logstash_minus_core.lib.logstash.pipeline.RUBY$method$maybe_setup_out_plugins$0$__VARARGS__(usr/share/logstash/logstash_minus_core/lib/logstash//usr/share/logstash/logstash-core/lib/logstash/pipeline.rb)", "usr.share.logstash.logstash_minus_core.lib.logstash.pipeline.start_workers(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:280)", "usr.share.logstash.logstash_minus_core.lib.logstash.pipeline.RUBY$method$start_workers$0$__VARARGS__(usr/share/logstash/logstash_minus_core/lib/logstash//usr/share/logstash/logstash-core/lib/logstash/pipeline.rb)", "usr.share.logstash.logstash_minus_core.lib.logstash.pipeline.run(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:217)", "usr.share.logstash.logstash_minus_core.lib.logstash.pipeline.RUBY$method$run$0$__VARARGS__(usr/share/logstash/logstash_minus_core/lib/logstash//usr/share/logstash/logstash-core/lib/logstash/pipeline.rb)", "usr.share.logstash.logstash_minus_core.lib.logstash.pipeline.start(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:176)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:295)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:274)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:270)", "java.lang.Thread.run(java/lang/Thread.java:829)"], :thread=>"#<Thread:0x6b3a135e run>"}
[2023-01-01T16:41:03,380][ERROR][logstash.agent ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}
I'm not sure if it's the true way of using ca file to validate ssl certificate among elastic products but this was the way I understood from the docs.
the pipeline structure is like the below:
input {
file {
...
}
}
filter {
...
}
output {
elasticsearch {
hosts => [ "https://*.*.*.*:9200" ]
cacert => "/etc/logstash/file_to_ES_input/elastic-stack-ca.p12"
user => elastic
password => "***"
index => "sample-files"
}
}
I would appreciate any helps for this issue.