Logstash - Invalid version of beats error

Elastic Stack Logstash
1

Hello everyone,

I have a Logstash server that receives data from Elastic agents and writes it to Elasticsearch. For testing purposes, I have one standalone agent running on a Linux server and one agent managed by a fleet server running on a Windows server. Both agents, the Logstash server, and the fleet are all version 8.11.3.

When the standalone agent writes to the Logstash server, everything works fine.
When the Windows agent writes to the Logstash server (configured as an Output via the fleet), I receive the following errors:

Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats: 3
An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats: 22
An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.

The errors come from pipeline "metrics", which is configured as follows:

input {
  elastic_agent {
  port => 5044
  }
}
output {
  pipeline {
  send_to => ["output_pipeline"]
  }
}
2

I had same problem when I configured client with SSL and logstash without.
Check if client isnt sending logs via SSL

3

I don't want to use SSL. I want the client to send data to the Logstash server without SSL, and for the Logstash server to write it to the Elasticsearch cluster without SSL. Is there no way to do that?

4

Yes there is a way.
I dont know the tool you are using but in filebeat it is:

f.e.

filebeat.yml:

output.logstash:
  hosts: ["logstash-node1:5044","logstash-node2:5044"]
  loadbalance: true

and in logstash

input {
  beats {
    port => 5044
    ssl_enabled => false
  }
}

output {
  elasticsearch {
    hosts => ["elk-node1:9200","elk-node2:9200"]
    index => "%{[@metadata][beat]}-%{+YYYY.ww}"
    }
}

it should work

5

I'm not using Filebeat directly but rather the Elastic Agent.
Additionally, I am not configuring the agent manually but through the Fleet server. I saw that on the Fleet server itself, it's not possible to configure Logstash as an output without SSL. How can I work around this?

6

How did you configure this output? Please share a screenshot of your Fleet UI showing the configuration.

This error of Invalid version of beats normally happens when you have something sending data to the beats/elastic_agent output without using the beats protocol.

How did you configure the host for your logstash in the logstash output in the Fleet UI?

I'm not sure this is related to ssl as according to the documentation ssl is disabled by default in both the beats input and also in the elastic_agent input, so you should not get any ssl errors because it is disable by default.

You can explicitly force it to be disabled using ssl_enabled => false in the elastic_agent input as well, same as the beats input.

How did you configure the output then?

This is required even if you are not going to use SSL, you would need to provide some dumb certificate for the client certificate and key, any dumb text would work.

I had the same issue a couple of months ago and opened an issue about it since it is possible to configure the output without SSL using the API, but not using the Fleet UI.

7

First of all, thank you for the assistance.

  1. I have only one input, which is elastic_agent, because I work with it. Do I need to use the beats input?
  2. In the Fleet GUI, I configured the output with the Logstash server name and port at the end, as you wrote instead of the SSL Certificate, I entered the string 'XXX'. Meanwhile, on the Logstash server, under the elastic_agent input, I configured: ssl_enabled => false

When the agent sends data to the Logstash server, I still get the error.

8

Handled by regenerating keys and certificates for the client and server. Unfortunately, I did not find an option to work without SSL.