Logstash-input-jms problem reading messages from IBM WebSphere MQ (missing valuefier)

Elastic Stack Logstash
1

I have problem getting messages from IBM WebSphere MQ. It seems that some field(s) in MQMD header has type which is not handled properly. Some messages are read without exception.
When I turn off include_header setting in jms input section, all messages are handled without any errors.
Exception is: java.lang.IllegalArgumentException: Missing Valuefier handling for full class name=java.lang.Byte, simple name=Byte

Logstash version: 5.3.0
Plugin version: logstash-input-jms-3.0.1-java
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
OS: Windows 7 Ent SP1, Linux RedHat (uname -r output: 2.6.32-642.4.2.el6.x86_64)
WebSphere MQ versions 7.5.0.6, 8.0.0.6

Config input section:

input {
    jms {
        yaml_file => "C:/logstash-5.3.0/config/jms.yml"
        yaml_section => "wmq"
        include_header => true
        include_properties => true
        include_body => true
        use_jms_timestamp => false
        destination => "TEST.LOG"
        interval => 3
        runner => async
        add_field => [ "host", "computer_name" ]
    }
}

jms.yml:
---
# IBM WebSphere MQ
wmq:
  :factory: com.ibm.mq.jms.MQQueueConnectionFactory
  :queue_manager: QMNAME
  :host_name: server_name.domain_name
  :channel: LOGSTASH.CHL
  :port: 1414
  # Transport Type: com.ibm.mq.jms.JMSC::MQJMS_TP_CLIENT_MQ_TCPIP
  :transport_type: 1
  :username: test
  :require_jars:
    - C:/IBM/MQ/java/lib/com.ibm.mqjms.jar

I can provide additional details if it is needed.

How to reproduce.
I can attach two sample messages which can be loaded to MQ queue using RFHUTIL (SupportPac IH03) (use "Load Q" button with default settings). One of these messages is processed without error, another message causes exception.

2

16:16:47.499 [DispatchThread: com.ibm.mq.jmqi.remote.impl.RemoteSession[connectionId=414D5143514D4D534B5630323120202058D31E1625C9467C]]] ERROR logstash.inputs.jms - Failed to create event {:message=>Java::ComIbmJms::JMSBytesMessage:Attributes: {:jms_correlation_id=>nil, :jms_delivery_mode_sym=>:persistent, :jms_destination=>nil, :jms_expiration=>0, :jms_message_id=>"ID:414d5120514d4d534b5630323120202058d31e1625c89c43", :jms_priority=>4, :jms_redelivered=>false, :jms_reply_to=>{"CCSID"=>1208, "brokerCCDurSubQueue"=>"SYSTEM.JMS.D.CC.SUBSCRIBER.QUEUE", "JMS_IBM_Unmappable_Action"=>"REPORT", "failIfQuiesce"=>1, "destDescription"=>nil, "readAheadClosePolicy"=>2, "JMS_IBM_Unmappable_Replacement"=>63, "mdWriteEnabled"=>false, "XMSC_WMQ_QUEUE_MANAGER"=>"QMNAME", "timeToLive"=>-2, "putAsyncAllowed"=>-1, "deliveryMode"=>-2, "readAheadAllowed"=>-1, "XMSC_DESTINATION_NAME"=>"REPLY.Q.NAME", "wildcardFormat"=>0, "mdReadEnabled"=>false, "messageBody"=>2, "XMSC_WMQ_BROKER_PUBQ"=>"","XMSC_ADMIN_OBJECT_TYPE"=>33, "priority"=>-2, "encoding"=>273, "brokerVersion"=>0, "version"=>7, "brokerDurSubQueue"=>"SYSTEM.JMS.D.SUBSCRIBER.QUEUE", "XMSC_CONNECTION_TYPE_NAME"=>"com.ibm.msg.client.wmq", "XMSC_WMQ_REPLYTO_STYLE"=>0, "targetClient"=>0, "mdMessageContext"=>0, "XMSC_WMQ_BROKER_PUBQ_QMGR"=>"", "XMSC_CONNECTION_TYPE"=>1}, :jms_timestamp=>1491916141150, :jms_type=>"mcd://xmlnsc/msg_set"}Properties: {"LOG_EventType"=>"REQFINISH", "JMS_IBM_Character_Set"=>"UTF-8", "JMS_IBM_Encoding"=>529, "ESB_Sndr"=>"Adp_name", "JMS_IBM_PutApplType"=>11, "ESB_Rcvr"=>"Srv_name", "JMS_IBM_Format"=>" ", "JMSXDeliveryCount"=>1, "LOG_ReplyQueue"=>"REPLY.Q.NAME", "JMS_IBM_PutTime"=>"13090115", "LOG_EventDate"=>"2017-04-06 15:42:20.532", "LOG_TargetQM"=>"QMNAME", "ESB_TrnId"=>"79acc522-1ac6-11e7-9350-0af37b6b0000", "ESB_MsgFlags"=>"0000000010", "JMS_IBM_MsgType"=>1, "ESB_MsgType"=>"REQUEST", "LOG_ServiceName"=>"Adp_name", "JMSXUserID"=>"ruamdaj ", "ESB_EntryPoint"=>"ADAPTER", "JMSXAppID"=>":\\IBM\\RFHUTIL75\\rfhutilc.exe", "ESB_BrkName"=>"BKNAME", "ESB_ExecGrp"=>"execgrp", "ESB_Operation"=>"GetInfo", "JMS_IBM_PutDate"=>"20170411", "ESB_MsgStatus"=>"SUCCESS", "LOG_TargetQueue"=>"TARGET.Q.NAME", "ESB_ESBId"=>"79acc522-1ac6-11e7-9350-0af37b6b0000"}, :exception=>java.lang.IllegalArgumentException: Missing Valuefier handling for full class name=java.lang.Byte, simple name=Byte, :backtrace=>["org.logstash.Valuefier.convertNonCollection(Valuefier.java:51)", "org.logstash.Valuefier.convert(Valuefier.java:90)", "org.logstash.ConvertedMap.newFromMap(ConvertedMap.java:31)", "org.logstash.Valuefier.convert(Valuefier.java:75)", "org.logstash.ext.JrubyEventExtLibrary$RubyEvent.ruby_set_field(JrubyEventExtLibrary.java:128)", "org.logstash.ext.JrubyEventExtLibrary$RubyEvent$INVOKER$i$2$0$ruby_set_field.call(JrubyEventExtLibrary$RubyEvent$INVOKER$i$2$0$ruby_set_field.gen)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:202)", "org.jruby.ast.CallTwoArgNode.interpret(CallTwoArgNode.java:59)", "org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)", "org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)", "org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:206)", "org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:157)", "org.jruby.runtime.Block.yield(Block.java:142)", "org.jruby.RubyHash$13.visit(RubyHash.java:1355)", "org.jruby.RubyHash.visitLimited(RubyHash.java:648)", "org.jruby.RubyHash.visitAll(RubyHash.java:634)", "org.jruby.RubyHash.iteratorVisitAll(RubyHash.java:1306)", "org.jruby.RubyHash.each_pairCommon(RubyHash.java:1351)", "org.jruby.RubyHash.each19(RubyHash.java:1342)", "org.jruby.RubyHash$INVOKER$i$0$0$each19.call(RubyHash$INVOKER$i$0$0$each19.gen)", "org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:143)", "org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:154)", "org.jruby.ast.CallNoArgBlockNode.interpret(CallNoArgBlockNode.java:64)", "org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)", "org.jruby.ast.IfNode.interpret(IfNode.java:116)", "org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)", "org.jruby.ast.BlockNode.interpret(BlockNode.java:71)", "org.jruby.ast.RescueNode.executeBody(RescueNode.java:221)", "org.jruby.ast.RescueNode.interpret(RescueNode.java:116)", "org.jruby.ast.BeginNode.interpret(BeginNode.java:83)", "org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)", "org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)", "org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:225)", "org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:219)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:202)", "org.jruby.ast.FCallTwoArgNode.interpret(FCallTwoArgNode.java:38)", "org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)", "org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)", "org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:206)", "org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:194)", "org.jruby.runtime.Interpreted19Block.call(Interpreted19Block.java:125)", "org.jruby.runtime.Block.call(Block.java:101)", "org.jruby.RubyProc.call(RubyProc.java:300)", "org.jruby.RubyProc.call19(RubyProc.java:281)", "org.jruby.RubyProc$INVOKER$i$0$0$call19.call(RubyProc$INVOKER$i$0$0$call19.gen)", "org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:210)", "org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:206)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:168)", "org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)", "org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)", "org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)", "org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:206)", "org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:194)", "org.jruby.runtime.Interpreted19Block.call(Interpreted19Block.java:125)", "org.jruby.runtime.Block.call(Block.java:101)", "org.jruby.RubyProc.call(RubyProc.java:300)", "org.jruby.RubyProc.call19(RubyProc.java:281)", "org.jruby.RubyProc$INVOKER$i$0$0$call19.call(RubyProc$INVOKER$i$0$0$call19.gen)", "org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:210)", "org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:206)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:168)", "org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)", "org.jruby.ast.IfNode.interpret(IfNode.java:116)",
(the entire stack trace does not fit to message)

3
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.