Logstash Kafka Client Version 0.8 input and Elasticsearch-6.6 output

I need to find logstash which one will be compatible with Kafka Client Version 0.8 on input and Elasticsearch-6.6 on output

I have kafka cluster and Kafka Client Version 0.8 compatible with it
I'm running logstash-2.4 with logstash-input-kafka (2.0.9) and it successfully reads messages from kafka and writes them to ES

Now I need to update ES cluster to 6.6 version and logstash-6.6.1 is unable to connect my kafka cluster I tried 5.x also didn't work, 2.4 cannot write to ES-6.6

I tried to upgrade logstash-output-elasticsearch plugin to version 9.3.2 on logstash-2.4 but it failed to write to ES-6.6.1 with error message

Encountered a retryable error. Will Retry with exponential backoff  {:code=>400, :url=>"http://172.17.0.2:9200/_template/logstash/_bulk", :level=>:error}
Encountered a retryable error. Will Retry with exponential backoff  {:code=>400, :url=>"http://172.17.0.2:9200/_template/logstash/_bulk/_bulk", :level=>:error}
Encountered a retryable error. Will Retry with exponential backoff  {:code=>400, :url=>"http://172.17.0.2:9200/_template/logstash/_bulk/_bulk/_bulk", :level=>:error}
Encountered a retryable error. Will Retry with exponential backoff  {:code=>400, :url=>"http://172.17.0.2:9200/_template/logstash/_bulk/_bulk/_bulk/_bulk", :level=>:error}
Encountered a retryable error. Will Retry with exponential backoff  {:code=>400, :url=>"http://172.17.0.2:9200/_template/logstash/_bulk/_bulk/_bulk/_bulk/_bulk", :level=>:error}

Then I tried to downgrade logstash-input-kafka plugin on logstash-6.6.1 to old version which worked on my old logstash (logstash-input-kafka (2.0.9))
And I get error during install
See error below
What else can i do in manner to achieve this problem?
Thanks in advance.

bash-4.2$ ./bin/logstash-plugin install --version 2.0.9 logstash-input-kafka
Validating logstash-input-kafka-2.0.9
Installing logstash-input-kafka
Plugin version conflict, aborting
ERROR: Installation Aborted, message: Bundler could not find compatible versions for gem "logstash-core-plugin-api":
  In snapshot (Gemfile.lock):
    logstash-core-plugin-api (= 2.1.16)

  In Gemfile:
    logstash-devutils (= 1.3.5) java depends on
      logstash-core-plugin-api (<= 2.99, >= 2.0) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-azure_event_hubs (>= 0) java depends on
      logstash-core-plugin-api (~> 2.0) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-jdbc (>= 0) java depends on
      logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-kafka (= 2.0.9) java depends on
      logstash-core-plugin-api (~> 1.0) java

    logstash-core-plugin-api (>= 0) java

Running `bundle update` will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.