Logstash not start after upgrade to new version

Elastic Stack Logstash
1

was 7.10.2, upgrade to 7.16.1
but it not start:

21-12-14T17:31:17,721][ERROR][logstash.javapipeline    ][main] Pipeline error {:pipeline_id=>"main", :exception=>#<LogStash::ConfigurationError: Could not connect to a compatible version of Elasticsearch>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.2.3-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:247:in `block in healthcheck!'", "org/jruby/RubyHash.java:1415:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.2.3-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:240:in `healthcheck!'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.2.3-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:374:in `update_urls'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.2.3-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:89:in `update_initial_urls'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.2.3-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:83:in `start'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.2.3-java/lib/logstash/outputs/elasticsearch/http_client.rb:359:in `build_pool'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.2.3-java/lib/logstash/outputs/elasticsearch/http_client.rb:63:in `initialize'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.2.3-java/lib/logstash/outputs/elasticsearch/http_client_builder.rb:106:in `create_http_client'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.2.3-java/lib/logstash/outputs/elasticsearch/http_client_builder.rb:102:in `build'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.2.3-java/lib/logstash/plugin_mixins/elasticsearch/common.rb:34:in `build_client'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.2.3-java/lib/logstash/outputs/elasticsearch.rb:275:in `register'", "org/logstash/config/ir/compiler/OutputStrategyExt.java:131:in `register'", "org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:68:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:232:in `block in register_plugins'", "org/jruby/RubyArray.java:1821:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:231:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:589:in `maybe_setup_out_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:244:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:189:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:141:in `block in start'"], "pipeline.sources"=>["/usr/share/logstash/pipeline/logstash.conf"], :thread=>"#<Thread:0x37e693f2 run>"}

config is pretty simple

input {
  beats {
    port => 5044
    client_inactivity_timeout => 3600
  }
}


output {

  elasticsearch {
    hosts => ""
    user=> ""
    ilm_enabled => false
    password=> ""
    ssl => true
    ssl_certificate_verification => false
  }
  
}

Elasticsearch is 7.10( aws opensearch)

2

That is the issue, from version 7.13+ the elasticsearch output plugin in Logstash checks if it is an Elasticsearch distributed by Elastic, which is not your case.

You should install the third party opensearch output plugin or downgrade Logstash, you won't be able to use the elasticsearch output plugin with Opensearch in this version.

For more information you should check the Opensearch foruns, as Opensearch is not supported here.

1 Like
3

Hey i'm running into this issue as well. I was previously using logstash-oss 7.10.2 with opendistro 1.13.2. This ran fine, but when i upgrade due to the log4j issue, the OSS version is now telling me it doesn't like ES. When I tried the opensearch-output-logstash, I still ran into the version problem. The 7.10.2 version worked, does this version have the log4j vulnerability?

4

When I tried the opensearch-output-logstash, I still ran into the version problem

AFAICT, it is not quite a drop-in replacement. To use it, you will also need to modify your pipeline to use the opensearch output plugin.

5

hey you have a link on that configuration? Thanks for the heads up.

6

hey you have a link on that configuration?

Not exactly, as I have never used it. It is a fork of our elasticsearch output plugin, and largely has the same options that were available at the time of fork, plus whatever else they have added.

Given the elasticsearch output plugin is invoked with something like:

output {
  elasticsearch {
    # ... plugin options
  }
}

Invoking the opensearch output plugin would therefore be something like:

output {
  opensearch {
    # ... plugin options
  }
}
7

lol Yaauie, yeah i just figure this out, thanks to your hint! Works like a charm now.

8

@blaklabz1 @yaauie @Alexander_Popov @leandrojmp
We are also facing the similar issue. We have used docker to install plugin as below

FROM docker.elastic.co/logstash/logstash:${logstash_version}
RUN bin/logstash-plugin install logstash-output-datadog_metrics
RUN bin/logstash-plugin install ?? <Want to install Opensearch plugin>

I didn't find any opensearch plugin here

Could someone tell me how to install the logstash opensearch plugin for ES opendistro 1.13.3.

I tried changing my output plugin from Elasticsearch to opensearch as below but did not work. Still it showed the pipeline failed to load

output {
  opensearch {
`.......` 
  }
}
9

Opensearch and Opendistro are not supported here.

You won' t find the opensearch output plugin listed on the Elastic documentation page, it is an third party plugin, it is not made by elastic.

You should check in the opensearch site or look for a opensearch community.

10

I had the same issues as above, while going through the lots of posts/blogs and figured out and fixed log4j vulnerabilities issue with my latest opendistro Elasticsearch image with logstash-0ss image, I am running this in K8s,

this is my dockerfile for Elasticsearch of opendistro,

FROM amazon/opendistro-for-elasticsearch:1.13.3
RUN ./bin/elasticsearch-plugin install --batch repository-azure

this is for opensearch logstash-oss image,

FROM opensearchproject/logstash-oss-with-opensearch-output-plugin:7.16.2
RUN bin/logstash-plugin install logstash-output-opsgenie
RUN bin/logstash-plugin install logstash-output-opensearch

and here is the details about output opensearch plugin,

        output {
          opensearch {
            hosts => "${ELASTICSEARCH_HOST}"
            user => "${ELASTICSEARCH_USER}"
            password => "${ELASTICSEARCH_PASSWORD}"
            cacert => "/usr/share/logstash/certs/ca.crt"
            ssl => true
            manage_template => false
            index => "dlq-%{+YYYY.MM.dd}"
          }

whosoever is facing issues of logstash-oss image from "docker.elastic.co/logstash/logstash-oss:7.8.1" and trying to make it compatible with Elasticsearch image of "amazon/opendistro-for-Elasticsearch:1.13.3", above code snippet should resolve it.

I was getting errors in logstash about "ilm_enabled => false" parameter in output plugin, I had to remove that.

Hope above things will help and save others time....I have learned and fixed my issues by reading above discussion threads, thank you guys :slight_smile:

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.