Hello,
I've setup a basic filebeat to logstash infra, but I keep on getting this kind of error in logstash' logs:
Mar 19 10:42:33 logstash[5912]: [2021-03-19T10:42:33,892][INFO ][org.logstash.beats.BeatsHandler][http-in-and-process][input-beats] [local: 10.2.10.19:5047, remote: 10.2.78.30:41046] Handling exception: org.logstash.FieldReference$IllegalSyntaxException: Invalid FieldReference: `string[]` (caused by: org.logstash.FieldReference$IllegalSyntaxException: Invalid FieldReference: `string[]`)
Mar 19 10:42:33 logstash[5912]: [2021-03-19T10:42:33,899][WARN ][io.netty.channel.DefaultChannelPipeline][http-in-and-process][input-beats] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
Mar 19 10:42:33 logstash[5912]: org.logstash.FieldReference$IllegalSyntaxException: Invalid FieldReference: `string[]`
Mar 19 10:42:33 logstash[5912]: at org.logstash.FieldReference$StrictTokenizer.tokenize(FieldReference.java:312) ~[logstash-core.jar:?]
Mar 19 10:42:33 logstash[5912]: at org.logstash.FieldReference.parse(FieldReference.java:213) ~[logstash-core.jar:?]
Mar 19 10:42:33 logstash[5912]: at org.logstash.FieldReference.parseToCache(FieldReference.java:204) ~[logstash-core.jar:?]
Mar 19 10:42:33 logstash[5912]: at org.logstash.FieldReference.from(FieldReference.java:127) ~[logstash-core.jar:?]
Mar 19 10:42:33 logstash[5912]: at org.logstash.ConvertedMap.put(ConvertedMap.java:95) ~[logstash-core.jar:?]
Mar 19 10:42:33 logstash[5912]: at org.logstash.ConvertedMap.newFromMap(ConvertedMap.java:75) ~[logstash-core.jar:?]
Mar 19 10:42:33 logstash[5912]: at org.logstash.Valuefier.lambda$initConverters$13(Valuefier.java:172) ~[logstash-core.jar:?]
Mar 19 10:42:33 logstash[5912]: at org.logstash.Valuefier.convert(Valuefier.java:94) ~[logstash-core.jar:?]
Mar 19 10:42:33 logstash[5912]: at org.logstash.ConvertedMap.newFromMap(ConvertedMap.java:75) ~[logstash-core.jar:?]
Mar 19 10:42:33 logstash[5912]: at org.logstash.Valuefier.lambda$initConverters$13(Valuefier.java:172) ~[logstash-core.jar:?]
Mar 19 10:42:33 logstash[5912]: at org.logstash.Valuefier.convert(Valuefier.java:94) ~[logstash-core.jar:?]
Mar 19 10:42:33 logstash[5912]: at org.logstash.ConvertedMap.newFromMap(ConvertedMap.java:75) ~[logstash-core.jar:?]
Mar 19 10:42:33 logstash[5912]: at org.logstash.Valuefier.lambda$initConverters$13(Valuefier.java:172) ~[logstash-core.jar:?]
Mar 19 10:42:33 logstash[5912]: at org.logstash.Valuefier.convert(Valuefier.java:94) ~[logstash-core.jar:?]
Mar 19 10:42:33 logstash[5912]: at org.logstash.ConvertedMap.newFromMap(ConvertedMap.java:75) ~[logstash-core.jar:?]
Mar 19 10:42:33 logstash[5912]: at org.logstash.ext.JrubyEventExtLibrary$RubyEvent.initializeFallback(JrubyEventExtLibrary.java:321) ~[logstash-core.jar:?]
Mar 19 10:42:33 logstash[5912]: at org.logstash.ext.JrubyEventExtLibrary$RubyEvent.ruby_initialize(JrubyEventExtLibrary.java:97) ~[logstash-core.jar:?]
Mar 19 10:42:33 logstash[5912]: at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_beats_minus_6_dot_0_dot_14_minus_java.lib.logstash.inputs.beats.message_listener.RUBY$method$onNewMessage$0(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-beats-6.0.14-java/lib/logstash/inputs/beats/message_listener.rb:40) ~[?:?]
Mar 19 10:42:33 logstash[5912]: at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_beats_minus_6_dot_0_dot_14_minus_java.lib.logstash.inputs.beats.message_listener.RUBY$method$onNewMessage$0$__VARARGS__(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-beats-6.0.14-java/lib/logstash/inputs/beats/message_listener.rb) ~[?:?]
Mar 19 10:42:33 logstash[5912]: at org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80) ~[jruby-complete-9.2.13.0.jar:?]
Mar 19 10:42:33 logstash[5912]: at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70) ~[jruby-complete-9.2.13.0.jar:?]
Mar 19 10:42:33 logstash[5912]: at org.jruby.gen.LogStash$$Inputs$$Beats$$MessageListener_1633205064.onNewMessage(org/jruby/gen/LogStash$$Inputs$$Beats$$MessageListener_1633205064.gen:13) ~[?:?]
Mar 19 10:42:33 logstash[5912]: at org.logstash.beats.BeatsHandler.channelRead0(BeatsHandler.java:52) ~[logstash-input-beats-6.0.14.jar:?]
Mar 19 10:42:33 logstash[5912]: at org.logstash.beats.BeatsHandler.channelRead0(BeatsHandler.java:12) ~[logstash-input-beats-6.0.14.jar:?]
Mar 19 10:42:33 logstash[5912]: at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
Mar 19 10:42:33 logstash[5912]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
Mar 19 10:42:33 logstash[5912]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
Mar 19 10:42:33 logstash[5912]: at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
Mar 19 10:42:33 logstash[5912]: at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
Mar 19 10:42:33 logstash[5912]: at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
Mar 19 10:42:33 logstash[5912]: at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
Mar 19 10:42:33 logstash[5912]: at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:61) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
Mar 19 10:42:33 logstash[5912]: at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:370) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
Mar 19 10:42:33 logstash[5912]: at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:66) ~[netty-all-4.1.49.Final.jar:4.1.49.Final]
Mar 19 10:42:33 logstash[5912]: at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-all-4.1.49.Final.jar:4.1.49.Final]
Mar 19 10:42:33 logstash[5912]: at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-all-4.1.49.Final.jar:4.1.49.Final]
Mar 19 10:42:33 logstash[5912]: at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-all-4.1.49.Final.jar:4.1.49.Final]
Mar 19 10:42:33 logstash[5912]: at java.lang.Thread.run(Thread.java:834) [?:?]
Mar 19 10:42:33 logstash[5912]: [2021-03-19T10:42:33,914][INFO ][org.logstash.beats.BeatsHandler][http-in-and-process][input-beats] [local: 10.2.10.19:5047, remote: 10.2.78.30:41046] Handling exception: org.jruby.exceptions.NoMethodError: (NoMethodError) undefined method `accept' for nil:NilClass (caused by: org.jruby.exceptions.NoMethodError: (NoMethodError) undefined method `accept' for nil:NilClass)
Mar 19 10:42:33 logstash[5912]: [2021-03-19T10:42:33,918][WARN ][io.netty.channel.DefaultChannelPipeline][http-in-and-process][input-beats] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
Mar 19 10:42:33 logstash[5912]: org.jruby.exceptions.NoMethodError: (NoMethodError) undefined method `accept' for nil:NilClass
Mar 19 10:42:33 logstash[5912]: at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_beats_minus_6_dot_0_dot_14_minus_java.lib.logstash.inputs.beats.message_listener.onNewMessage(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-beats-6.0.14-java/lib/logstash/inputs/beats/message_listener.rb:44) ~[?:?]
Input configuration for this pipeline:
input {
http {
id => "input-http"
port => 5046
host => "0.0.0.0"
codec => "json"
}
beats {
id => "input-beats"
port => 5047
type => "beats"
client_inactivity_timeout => 1200
}
}
Logstash 7.11 logstash/stable,now 1:7.11.2-1 amd64 [installed]
Filebeat config:
output.logstash:
hosts:
- "host:5047"
worker: 48
ttl: 30
bulk_max_size: 4096
compression_level: 4
loadbalance: false
Did I do something wrong here ?