Logstash-input-jms and solace issue

Elastic Stack Logstash
1

I am trying to connect the logstash jms input plugin to a Solace PubSub topic with no luck.

The resources I have used:

To test the solace instance I used the java-api:

My configuration files for logstash:

logstash-jms.conf

input {
  jms {
    yaml_section => "solace"
    include_header => true
    include_properties => true
    include_body => true
    use_jms_timestamp => false
    yaml_file => "/path/to/jms.yml"
    destination => "tutorial/topic"
    pub_sub => true
  }
}

output {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "test_jms"
  }
  stdout {
    codec => rubydebug
  }
}

jms.yml

# Solace
solace:
  :jndi_name: jms/defaultConnectionFactory
  :factory: com.solacesystems.jndi.SolJNDIInitialContextFactory
  :host_name: <IP>
  :channel: LOGSTASH.CHL
  :port: 55555
  :transport_type: 1
  :username: ****
  :jndi_context:
    java.naming.factory.initial: com.solacesystems.jndi.SolJNDIInitialContextFactory
    java.naming.provider.url: smf://<IP>:55555
    java.naming.security.principal: ****
    java.naming.security.credentials: ****
  :require_jars:
    - /path/to/commons-lang-2.6.jar
    - /path/to/commons-logging-1.1.3.jar
    - /path/to/geronimo-jms_1.1_spec-1.1.1.jar
    - /path/to/sol-common-10.3.0.jar
    - /path/to/sol-jcsmp-10.3.0.jar
    - /path/to/sol-jms-10.3.0.jar

The error message

$ ./bin/logstash -f logstash-jms.conf 
Sending Logstash's logs to /path/to/logs which is now configured via log4j2.properties
[2018-05-30T15:35:23,418][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"/path/to/modules/netflow/configuration"}
[2018-05-30T15:35:23,442][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"/path/to/modules/fb_apache/configuration"}
[2018-05-30T15:35:23,671][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-05-30T15:35:24,228][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.2.3"}
[2018-05-30T15:35:24,681][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2018-05-30T15:35:28,247][INFO ][logstash.pipeline        ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2018-05-30T15:35:28,715][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}}
[2018-05-30T15:35:28,727][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"}
[2018-05-30T15:35:28,947][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://localhost:9200/"}
[2018-05-30T15:35:29,035][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>6}
[2018-05-30T15:35:29,040][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>6}
[2018-05-30T15:35:29,062][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2018-05-30T15:35:29,103][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2018-05-30T15:35:29,153][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]}
[2018-05-30T15:35:29,470][INFO ][logstash.pipeline        ] Pipeline started succesfully {:pipeline_id=>"main", :thread=>"#<Thread:0x68ede160 run>"}
[2018-05-30T15:35:29,571][INFO ][logstash.agent           ] Pipelines running {:count=>1, :pipelines=>["main"]}
[2018-05-30T15:35:29,738][WARN ][logstash.inputs.jms      ] JMS Consumer died {:exception=>#<NoMethodError: undefined method `create_connection' for #<Java::ComSolacesystemsJndi::SolJNDIInitialContextFactory:0x13a17dd3>>, :backtrace=>["/path/to/vendor/bundle/jruby/2.3.0/gems/jruby-jms-1.2.0-java/lib/jms/connection.rb:202:in `initialize'", "/path/to/vendor/bundle/jruby/2.3.0/gems/jruby-jms-1.2.0-java/lib/jms/connection.rb:50:in `start'", "/path/to/vendor/bundle/jruby/2.3.0/gems/jruby-jms-1.2.0-java/lib/jms/connection.rb:70:in `session'", "/path/to/vendor/bundle/jruby/2.3.0/gems/logstash-input-jms-3.0.5-java/lib/logstash/inputs/jms.rb:186:in `run_consumer'", "/path/to/vendor/bundle/jruby/2.3.0/gems/logstash-input-jms-3.0.5-java/lib/logstash/inputs/jms.rb:257:in `run'", "/path/to/logstash-core/lib/logstash/pipeline.rb:516:in `inputworker'", "/path/to/logstash-core/lib/logstash/pipeline.rb:509:in `block in start_input'"]}

The exception

undefined method `create_connection'

Thanks

3

I have a fix for the problem above:

Change logstash-jms.yml to

input {
  jms {
    yaml_section => "solace"
    include_header => true
    include_properties => true
    include_body => true
    use_jms_timestamp => false
    yaml_file => "/path/to/jms.yml"
    destination => "tutorial/topic"
    pub_sub => true
    runner => "async"
  }
}

output {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "test_jms"
  }
  stdout {
    codec => rubydebug
  }
}

And change jms.yml to

# Solace
solace:
  :factory: com.solacesystems.jms.SolConnectionFactoryImpl
  :port: 55555
  :host: <IP>
  :transport_type: 1
  :username: ****
  :password: ****
  :require_jars:
    - /path/to/commons-lang-2.6.jar
    - /path/to/commons-logging-1.1.3.jar
    - /path/to/geronimo-jms_1.1_spec-1.1.1.jar
    - /path/to/sol-common-10.3.0.jar
    - /path/to/sol-jcsmp-10.3.0.jar
    - /path/to/sol-jms-10.3.0.jar

I could not however get it to work with the JNDI lookup technique.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.