Logstash 7.7 can’t establish pipeline with ssl

Elastic Stack Logstash
1

Hi Everyone,

So I have been trying to push data from logstash to Elastic using ssl. I am facing the issue when I am doing this thing on Logstash which is running on 7.7 and When I am doing same configuration on Logstash which is running on 8.14 things are working fine.

Logstash Config :

    if "abc" in [index_name] {
        if "_customfailure" in [tags] {
            elasticsearch {
                index => "failure_index_name_%{+yyyy_MM_dd}"
                hosts => ["https://es-node1:9200","https://es-node2:9200","https://es-node3:9200"]
                user => "username"
                password => "password"
                ssl => true
                cacert => "/opt/elastic_certs/certs/http_ca.crt"
            }
        }
        else {
            elasticsearch {
                index => "index_name%{[@metadata][index_name_day]}"
                hosts => ["https://es-node1:9200","https://es-node2:9200","https://es-node3:9200"]
                user => "username"
                password => "password"
                ssl => true
                cacert => "/opt/elastic_certs/certs/http_ca.crt"
                retry_initial_interval => 1
                retry_max_interval => 1
                retry_on_conflict => 1
            }
        }
    }

Error :

[2024-07-12T08:20:31,286][ERROR][logstash.javapipeline    ][main] Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<Manticore::UnknownException: Unsupported or unrecognized SSL message>, :backtrace=>["/home/ec2-user/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:37:in `block in initialize'", "/home/ec2-user/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/response.rb:79:in `call'", "/home/ec2-user/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.5.1-java/lib/logstash/outputs/elasticsearch/http_client/manticore_adapter.rb:74:in `perform_request'", "/home/ec2-user/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.5.1-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:332:in `perform_request_to_url'", "/home/ec2-user/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.5.1-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:261:in `health_check_request'", "/home/ec2-user/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.5.1-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:270:in `block in healthcheck!'", "org/jruby/RubyHash.java:1415:in `each'", "/home/ec2-user/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.5.1-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:266:in `healthcheck!'", "/home/ec2-user/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.5.1-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:382:in `update_urls'", "/home/ec2-user/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.5.1-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:82:in `update_initial_urls'", "/home/ec2-user/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.5.1-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:76:in `start'", "/home/ec2-user/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.5.1-java/lib/logstash/outputs/elasticsearch/http_client.rb:302:in `build_pool'", "/home/ec2-user/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.5.1-java/lib/logstash/outputs/elasticsearch/http_client.rb:64:in `initialize'", "/home/ec2-user/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.5.1-java/lib/logstash/outputs/elasticsearch/http_client_builder.rb:105:in `create_http_client'", "/home/ec2-user/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.5.1-java/lib/logstash/outputs/elasticsearch/http_client_builder.rb:101:in `build'", "/home/ec2-user/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.5.1-java/lib/logstash/outputs/elasticsearch.rb:274:in `build_client'", "/home/ec2-user/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.5.1-java/lib/logstash/outputs/elasticsearch/common.rb:23:in `register'", "org/logstash/config/ir/compiler/OutputStrategyExt.java:126:in `register'", "org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:68:in `register'", "/home/ec2-user/logstash/logstash-core/lib/logstash/java_pipeline.rb:216:in `block in register_plugins'", "org/jruby/RubyArray.java:1809:in `each'", "/home/ec2-user/logstash/logstash-core/lib/logstash/java_pipeline.rb:215:in `register_plugins'", "/home/ec2-user/logstash/logstash-core/lib/logstash/java_pipeline.rb:520:in `maybe_setup_out_plugins'", "/home/ec2-user/logstash/logstash-core/lib/logstash/java_pipeline.rb:228:in `start_workers'", "/home/ec2-user/logstash/logstash-core/lib/logstash/java_pipeline.rb:170:in `run'", "/home/ec2-user/logstash/logstash-core/lib/logstash/java_pipeline.rb:125:in `block in start'"], "pipeline.sources"=>["/home/ec2-user/logstash/config/configs/log_monitoring_logstash.conf"], :thread=>"#<Thread:0x1be87230 run>"}
[2024-07-12T08:20:31,301][ERROR][logstash.agent           ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}

Note : same thing is working with Logstash-8.14

2

Anyone here who faced similar issue or if anyone can help me in this.

3

Are you sure that was working with the cacert path outside /etc/elasticsearch?