Reading from jms with input-jms

Elastic Stack Logstash
1

guys hello,

i test to read message from websphere mq,
logstash 5.5.2 - windows 8
elasticsearch 5.5.2 - linux
websphere mq 8.0.0.4

i have 2 test case:
#1) with filter in logstash.conf
#2) without filter

#1) average logstash handle 215 message in second
#2) avergate logstash handle 350 message in second

is it possible to increase throughput?

jms.yml
# IBM WebSphere MQ
wmq:
:factory: com.ibm.mq.jms.MQQueueConnectionFactory
:queue_manager: QMTEST
:host_name: 192.168.0.181
:channel: LOGSTASH.CHL
:port: 1414
# Transport Type: com.ibm.mq.jms.JMSC::MQJMS_TP_CLIENT_MQ_TCPIP
:transport_type: 1
:username: Test
:require_jars:
- "C:/distr/logstash-5.6.2/jms/jms-1.0.2b.jar"
- "C:/distr/logstash-5.6.2/jms/com.ibm.mq.jar"
- "C:/distr/logstash-5.6.2/jms/com.ibm.mqjms.jar"
- "C:/distr/logstash-5.6.2/jms/com.ibm.mq.allclient.jar"

logstash_jms.conf

nugusbayevkk@mediator:~/Downloads$ cat logstash_jms.conf
input {
jms {
yaml_file => "C:/distr/logstash-5.6.2/jms/jms.yml"
yaml_section => "wmq"
include_header => false
include_properties => false
include_body => true
use_jms_timestamp => false
destination => "TEST.LOG"
interval => 3
runner => async
add_field => [ "host", "computer_name" ]
}
}

filter {
xml {
source => message
store_xml => false
xpath => {
"//RqUID/text()|//MsgID/text()|//RqID/text()" => "RqUID"
"//Operation/text()" => "operation"
"//EventId/text()" => "event_id"
"//EventMsg" => "event_msg"
"//EventDate/text()" => "event_date"
"//ProcStatus/text()" => "proc_status"
"//ErrorText/text()" => "error_text"
"//EventSource/text()" => "event_source"
"//EventReceiver/text()" => "event_receiver"
"//SourceQueue/text()" => "source_queue"
"//SourceQM/text()" => "source_qm"
"//TargetQueue/text()" => "target_queue"
"//TargetQM/text()" => "target_qm"
"//ReplyQueue/text()" => "reply_queue"
"//ReplyQM/text()" => "reply_qm"
}

}

mutate {
remove_field => ["headers"]
}

}

output {
elasticsearch {
hosts => ["192.168.0.171:9200"]
index => "bus-%{+YYYY.MM.dd}"
}
}

message string size about 1000.

2

Have you tried varying the number of threads for the input plugin to see if that makes any difference?

3

ok, thanks Christian, after testing cases with threading i will write my results.

4

3# case: 4 threads - about 253 handled message/sec with filter
4# case: 3 threads - about 258 handled message/sec with filter

Christian thanks, here i tested on my notebooks, and i use wifi-network, so i think on test, or production environment, throughput must increase.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.