Logstash 7.17.2 and Elastic 7.10.2 (latest repo version) error: could not connect to compatible elastic version

Hi all,

So yesterday we installed logstash and Elasticsearch via yum from repo:
name=Elasticsearch repository for 7.x packages baseurl=https://artifacts.elastic.co/packages/oss-7.x
(forgive me if a misuse terms, I am a software developer, not a systems engineer) on a linux server.

Installed versions:

Installed Packages
Name        : elasticsearch-oss
Arch        : x86_64
Version     : 7.10.2
Release     : 1
Size        : 401 M
Repo        : installed
From repo   : elasticsearch-7.x
Summary     : Distributed RESTful search engine built for the cloud
URL         : https://www.elastic.co/
License     : ASL 2.0
Installed Packages
Name        : logstash-oss
Arch        : x86_64
Epoch       : 1
Version     : 7.17.2
Release     : 1
Size        : 596 M
Repo        : installed
From repo   : elasticsearch-7.x
Summary     : An extensible logging pipeline
URL         : https://www.elastic.co/logstash
License     : ASL 2.0

Elastic is up and running:

curl "localhost:9200"
{
  "name" : "wntstfo03.izaaksuite.nl",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "wRQY0SfoTCuo5MtnxtJ_FA",
  "version" : {
    "number" : "7.10.2",
    "build_flavor" : "oss",
    "build_type" : "rpm",
    "build_hash" : "747e1cc71def077253878a59143c1f785afa92b9",
    "build_date" : "2021-01-13T00:42:12.435326Z",
    "build_snapshot" : false,
    "lucene_version" : "8.7.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"

But when starting logstash with following configuration:

input {
    jdbc {
        jdbc_connection_string => "jdbc:oracle:thin:@wntstdb03.izaaksuite.nl:1521:wntstf2"
        jdbc_user => "webnext_zaken"
		jdbc_password => "webnext_zaken"
        jdbc_driver_library => ""
        jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
        statement_filepath =>"/appl/sw/webnext/logstash/config_documenten/queries/documenten.sql"
	last_run_metadata_path => "/appl/sw/webnext/logstash/config_documenten/parameters/.jdbc_last_run_doc"
    }
}

# The filter part of this file is commented out to indicate that it is
# optional.

#filter {
#    
#}

output {
 stdout { codec => rubydebug }
  file { 
  path => ["/appl/sw/webnext/logstash/config_documenten/output/documenten.txt"]
 	}
  elasticsearch { 
 		hosts => ["localhost:9200"]
 		index => "documenten"
 		document_id => "%{zaakdoc_id}"
 	}
}

we get an error:

/usr/share/logstash/bin/logstash "--path.settings" "/appl/sw/webnext/logstash/config_documenten"
Using JAVA_HOME defined java: /usr/lib/jvm/java
WARNING: Using JAVA_HOME while Logstash distribution comes with a bundled JDK.
DEPRECATION: The use of JAVA_HOME is now deprecated and will be removed starting from 8.0. Please configure LS_JAVA_HOME instead.
Sending Logstash logs to /usr/share/logstash/logs which is now configured via log4j2.properties
[2022-04-05T11:25:51,834][INFO ][logstash.runner          ] Log4j configuration path used is: /appl/sw/webnext/logstash/config_documenten/log4j2.properties
[2022-04-05T11:25:51,892][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.17.2", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 25.302-b08 on 1.8.0_302-b08 +indy +jit [linux-x86_64]"}
[2022-04-05T11:25:51,895][INFO ][logstash.runner          ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -Djruby.regexp.interruptible=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true]
[2022-04-05T11:25:53,857][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[2022-04-05T11:25:55,938][INFO ][org.reflections.Reflections] Reflections took 105 ms to scan 1 urls, producing 119 keys and 419 values
[2022-04-05T11:25:59,827][INFO ][logstash.outputs.elasticsearch][documenten] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]}
[2022-04-05T11:26:00,360][INFO ][logstash.outputs.elasticsearch][documenten] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}}
[2022-04-05T11:26:01,830][ERROR][logstash.javapipeline    ][documenten] Pipeline error {:pipeline_id=>"documenten", :exception=>#<LogStash::ConfigurationError: Could not connect to a compatible version of Elasticsearch>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.4.1-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:245:in `block in healthcheck!'", "org/jruby/RubyHash.java:1415:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.4.1-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:238:in `healthcheck!'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.4.1-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:370:in `update_urls'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.4.1-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:87:in `update_initial_urls'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.4.1-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:81:in `start'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.4.1-java/lib/logstash/outputs/elasticsearch/http_client.rb:358:in `build_pool'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.4.1-java/lib/logstash/outputs/elasticsearch/http_client.rb:63:in `initialize'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.4.1-java/lib/logstash/outputs/elasticsearch/http_client_builder.rb:106:in `create_http_client'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.4.1-java/lib/logstash/outputs/elasticsearch/http_client_builder.rb:102:in `build'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.4.1-java/lib/logstash/plugin_mixins/elasticsearch/common.rb:34:in `build_client'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.4.1-java/lib/logstash/outputs/elasticsearch.rb:279:in `register'", "org/logstash/config/ir/compiler/OutputStrategyExt.java:131:in `register'", "org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:68:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:232:in `block in register_plugins'", "org/jruby/RubyArray.java:1821:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:231:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:589:in `maybe_setup_out_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:244:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:189:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:141:in `block in start'"], "pipeline.sources"=>["/appl/sw/webnext/logstash/config_documenten/documenten.conf"], :thread=>"#<Thread:0x6e88b789 run>"}
[2022-04-05T11:26:01,834][INFO ][logstash.javapipeline    ][documenten] Pipeline terminated {"pipeline.id"=>"documenten"}
[2022-04-05T11:26:01,860][ERROR][logstash.agent           ] Failed to execute action {:id=>:documenten, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<documenten>, action_result: false", :backtrace=>nil}
[2022-04-05T11:26:02,000][INFO ][logstash.runner          ] Logstash shut down.
[2022-04-05T11:26:02,015][FATAL][org.logstash.Logstash    ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.20.1.jar:?]
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.20.1.jar:?]
        at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:94) ~[?:?]

Any help is greatly appreciated!

If you need to run Elasticsearch 7.10 then downgrade your logstash to 7.10.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.