Error connecting Spring Microservices to elastic stack

Im using the logstash docker images for a "simple" integration with java spring cloud microservices,
at the end I made every container run smoothly, the problem came when I was trying to pass the logs from the microservice to kibana using , logstash and elasticSearch, the container of logstash marks this error :

[2020-04-14T23:20:20,699][INFO ][org.logstash.beats.BeatsHandler][main] [local: 192.168.176.6:5044, remote: 192.168.176.1:46522] Handling exception: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123
[2020-04-14T23:20:20,700][WARN ][io.netty.channel.DefaultChannelPipeline][main] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:472) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:38) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:353) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:66) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897) [netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-all-4.1.30.Final.jar:4.1.30.Final]
        at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123
        at org.logstash.beats.Protocol.version(Protocol.java:22) ~[logstash-input-beats-6.0.9.jar:?]
        at org.logstash.beats.BeatsParser.decode(BeatsParser.java:62) ~[logstash-input-beats-6.0.9.jar:?]
        at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
        ... 8 more
[2020-04-14T23:20:20,702][INFO ][org.logstash.beats.BeatsHandler][main] [local: 192.168.176.6:5044, remote: 192.168.176.1:46522] Handling exception: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34
[2020-04-14T23:20:20,702][WARN ][io.netty.channel.DefaultChannelPipeline][main] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:472) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:405) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:372) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:355) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.channel.AbstractChannelHandlerContext.access$300(AbstractChannelHandlerContext.java:38) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.channel.AbstractChannelHandlerContext$4.run(AbstractChannelHandlerContext.java:236) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:66) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897) [netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-all-4.1.30.Final.jar:4.1.30.Final]
        at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34
        at org.logstash.beats.Protocol.version(Protocol.java:22) ~[logstash-input-beats-6.0.9.jar:?]
        at org.logstash.beats.BeatsParser.decode(BeatsParser.java:62) ~[logstash-input-beats-6.0.9.jar:?]
        at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
        ... 10 more 

i been looking into another similar errors but they all mark different beats protocol, I'm using these docker compose file to invoke logstash:


logstash:
    image: logstash:7.6.2
    container_name: logstash
    ports:
      - 5044:5044
      - 9600:9600
    volumes:
      - ./elastic-stack/logstash/logstash.yml:/usr/share/logstash/config/logstash.yml
      - ./elastic-stack/logstash/pipeline:/usr/share/logstash/config/pipelines:ro
    environment:
      - LS_JAVA_OPTS=-Xmx256m -Xms256m
    networks:
      - iot-services
    depends_on:
      - elasticsearch

with these files:
Logstash.conf:


input {
  beats{
    tcp {
       port => 5044
       codec => "json"
       ssl => false
    }
  }
 }

 output {
     elasticsearch {
        hosts => ["elasticsearch:9300"]
        index => "micro-%{serviceName}"
    }
 }

logstash.yml:

http.host: "0.0.0.0"
path.config: /usr/share/logstash/pipeline
xpack.monitoring.enabled: false

and this conf.xml on microservices:


<appender name="STASH"
              class="net.logstash.logback.appender.LogstashTcpSocketAppender">
        <destination>0.0.0.0:5044</destination>

        <encoder
                class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
            <providers>
                <mdc /> <!-- MDC variables on the Thread will be written as JSON fields -->
                <context /> <!--Outputs entries from logback's context -->
                <version /> <!-- Logstash json format version, the @version field in the output -->
                <logLevel />
                <loggerName />

                <pattern>
                    <pattern>
                        {
                        "serviceName": "device-service"
                        }
                    </pattern>
                </pattern>

                <threadName />
                <message />
                <logstashMarkers />
                <stackTrace />
            </providers>
        </encoder>
    </appender>

following this tutorial:

what do you think it´s the problem? the SSL or just another configuration that I'm missing?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.