Shah_Zain
November 22, 2023, 9:35am
1
Logstash unable to receive data from MQTT. Getting this from logs:
//Stack: C:/Users/Shah Zain/Downloads/logstash-8.11.1-windows-x86_64/logstash-8.11.1/vendor/bundle/jruby/3.1.0/gems/logstash-input-mqtt-0.0.2/lib/logstash/inputs/mqtt.rb:63:in block in run' C:/Users/Shah Zain/Downloads/logstash-8.11.1-windows-x86_64/logstash-8.11.1/logstash-core/lib/logstash/codecs/delegator.rb:64:in block in decode'decode' C:/Users/Shah Zain/Downloads/logstash-8.11.1-windows-x86_64/logstash-8.11.1/logstash-core/lib/logstash/codecs/delegator.rb:62:in block in decode'time' org/logstash/instrument/metrics/AbstractNamespacedMetricExt.java:68:in time'decode' C:/Users/Shah Zain/Downloads/logstash-8.11.1-windows-x86_64/logstash-8.11.1/vendor/bundle/jruby/3.1.0/gems/logstash-input-mqtt-0.0.2/lib/logstash/inputs/mqtt.rb:62:in block in run'block in get' C:/Users/Shah Zain/Downloads/logstash-8.11.1-windows-x86_64/logstash-8.11.1/vendor/bundle/jruby/3.1.0/gems/mqtt-0.6.0/lib/mqtt/client.rb:424:in block in get_packet'loop' C:/Users/Shah Zain/Downloads/logstash-8.11.1-windows-x86_64/logstash-8.11.1/vendor/bundle/jruby/3.1.0/gems/mqtt-0.6.0/lib/mqtt/client.rb:422:in get_packet'get' C:/Users/Shah Zain/Downloads/logstash-8.11.1-windows-x86_64/logstash-8.11.1/vendor/bundle/jruby/3.1.0/gems/logstash-input-mqtt-0.0.2/lib/logstash/inputs/mqtt.rb:61:in run'inputworker' C:/Users/Shah Zain/Downloads/logstash-8.11.1-windows-x86_64/logstash-8.11.1/logstash-core/lib/logstash/java_pipeline.rb:405:in block in start_input'LogStash::Event:0x76884d8d
my config file
input {
mqtt {
port => 1883
topic => ["Application1/Message"]
qos => 0
}
}
filter {
}
output {
stdout { codec => rubydebug }
elasticsearch {
hosts => ["http://localhost:9200"]
index => "test.logstash"
user => "elastic"
password => ""
}
}
leandrojmp
November 22, 2023, 1:00pm
2
This plugin is a third-party plugin, it is not developed by Elastic.
It seems that development is stalled, the last commit was 5 years ago, as you can see in the repository .
It may be some incompatibility issue with newer versions, you need to contact directly the developer of the plugin.
Also, checking in the same repository , the topic option in the example input is a string, not an array like you configured, have you try to change it to topic => "Application1/Message" to see if it works?
Shah_Zain
November 22, 2023, 1:46pm
3
But the one i have downloaded is nov 2023 build. Download Logstash Free | Get Started Now | Elastic after changing to Application1/Message I am getting same error. but if i add mqttHost => "localhost" I get different error:
Plugin: <LogStash::Inputs::Mqtt topic=>"Application1/Message", mqttHost=>"localhost", id=>"9747185c6633272ffae6442affc5fadc9e6830b41fae12a56fc4752b5078bc5f", qos=>0, port=>1883, client_id=>"logstash_1", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_614ff558-2654-4f91-bffd-19dced545276", enable_metric=>true, charset=>"UTF-8">, ssl=>false, clean_session=>true>
Error: undefined method `[]' for #<LogStash::Event:0x1f361496>
Exception: NoMethodError
Stack: C:/Users/Shah Zain/Downloads/logstash-8.11.1-windows-x86_64/logstash-8.11.1/vendor/bundle/jruby/3.1.0/gems/logstash-input-mqtt-0.0.2/lib/logstash/inputs/mqtt.rb:63:in `block in run'
C:/Users/Shah Zain/Downloads/logstash-8.11.1-windows-x86_64/logstash-8.11.1/logstash-core/lib/logstash/codecs/delegator.rb:64:in `block in decode'
C:/Users/Shah Zain/Downloads/logstash-8.11.1-windows-x86_64/logstash-8.11.1/vendor/bundle/jruby/3.1.0/gems/logstash-codec-plain-3.1.0/lib/logstash/codecs/plain.rb:54:in `decode'
C:/Users/Shah Zain/Downloads/logstash-8.11.1-windows-x86_64/logstash-8.11.1/logstash-core/lib/logstash/codecs/delegator.rb:62:in `block in decode'
org/logstash/instrument/metrics/AbstractSimpleMetricExt.java:74:in `time'
org/logstash/instrument/metrics/AbstractNamespacedMetricExt.java:68:in `time'
C:/Users/Shah Zain/Downloads/logstash-8.11.1-windows-x86_64/logstash-8.11.1/logstash-core/lib/logstash/codecs/delegator.rb:61:in `decode'
C:/Users/Shah Zain/Downloads/logstash-8.11.1-windows-x86_64/logstash-8.11.1/vendor/bundle/jruby/3.1.0/gems/logstash-input-mqtt-0.0.2/lib/logstash/inputs/mqtt.rb:62:in `block in run'
C:/Users/Shah Zain/Downloads/logstash-8.11.1-windows-x86_64/logstash-8.11.1/vendor/bundle/jruby/3.1.0/gems/mqtt-0.6.0/lib/mqtt/client.rb:392:in `block in get'
C:/Users/Shah Zain/Downloads/logstash-8.11.1-windows-x86_64/logstash-8.11.1/vendor/bundle/jruby/3.1.0/gems/mqtt-0.6.0/lib/mqtt/client.rb:424:in `block in get_packet'
org/jruby/RubyKernel.java:1586:in `loop'
C:/Users/Shah Zain/Downloads/logstash-8.11.1-windows-x86_64/logstash-8.11.1/vendor/bundle/jruby/3.1.0/gems/mqtt-0.6.0/lib/mqtt/client.rb:422:in `get_packet'
C:/Users/Shah Zain/Downloads/logstash-8.11.1-windows-x86_64/logstash-8.11.1/vendor/bundle/jruby/3.1.0/gems/mqtt-0.6.0/lib/mqtt/client.rb:391:in `get'
C:/Users/Shah Zain/Downloads/logstash-8.11.1-windows-x86_64/logstash-8.11.1/vendor/bundle/jruby/3.1.0/gems/logstash-input-mqtt-0.0.2/lib/logstash/inputs/mqtt.rb:61:in `run'
C:/Users/Shah Zain/Downloads/logstash-8.11.1-windows-x86_64/logstash-8.11.1/logstash-core/lib/logstash/java_pipeline.rb:414:in `inputworker'
C:/Users/Shah Zain/Downloads/logstash-8.11.1-windows-x86_64/logstash-8.11.1/logstash-core/lib/logstash/java_pipeline.rb:405:in `block in start_input'
leandrojmp
November 22, 2023, 1:51pm
4
The Logstash version is unrelated, the mqtt input plugin is not made by Elastic, it is a third-party plugin that you can install in Logstash.
According to the repository of the plugin, the last time it was updated was 5 years ago, it may have some incompatibility issues with newer Logstash versions since it cannot start.
You need to contact the developer of the plugin.
1 Like
Shah_Zain
November 22, 2023, 2:03pm
5
That is strange, because we are using the same in production env and it is working. The error is present only when I configure it locally.
leandrojmp
November 22, 2023, 2:31pm
6
How did you install it?
I was looking into the spec of the plugin, and if you use logstash-plugin install logstash-input-mqtt, it will install this one , which is not the one I linked in a previous post.
But this plugin also does not have an update in years.
Since you said it is working in your production environment, what is the difference to your local environment? Is Logstash and the operating system in the same version?
Badger
November 22, 2023, 6:08pm
7
That looks very much like it is running this code. Although that is version 0.0.1 and it was updated in 0.0.2 (6 years ago) to be compatible with logstash version 5. You appear to be running 0.0.2, but that would not produce that error.
Either way, you will need to pick which of the two mqtt plugins you want to run and then update it to work with a current logstash version.
system
December 20, 2023, 6:08pm
8
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
