Mongodb jdbc using in logstash

trying to write log from mongodb using jdbc
see configuration file:

input {
  jdbc {
        jdbc_driver_library => "/usr/share/logstash/logstash-core/lib/jars/mongodb-driver-core-3.4.2.jar"
       	jdbc_driver_class => "Java::mongodb.jdbc.MongoDriver"
       	jdbc_connection_string => "jdbc:mongodb://<user><password>@<serverip>:27017/stg?authSource=admin"
	jdbc_user => "<user>"
    	jdbc_password => "<password>"
    	#schedule => "*/10 * * * * *"
       	statement => "db.logEvents.find({});"
  }
}
output {
 file {
    path => "/var/log/logstash/carobd-mongo.log"
 }
}

and i get an error in logstash log :

[2021-12-09T12:35:48,974][DEBUG][logstash.inputs.jdbc     ] loading /usr/share/logstash/logstash-core/lib/jars/mongodb-driver-core-3.4.2.jar
  Pipeline_id:mongo
  Plugin: <LogStash::Inputs::Jdbc jdbc_user=>"user", jdbc_password=><password>, statement=>"db.logEvents.find({});", jdbc_driver_library=>"/usr/share/logstash/logstash-core/lib/jars/mongodb-driver-core-3.4.2.jar", jdbc_connection_string=>"jdbc:mongodb://user:*****@<serverip>:27017/stg?authSource=admin", id=>"d978c12aa530ca4c0d00a4d513b141d959d66a05d400583b2e6046568d83c8b3", jdbc_driver_class=>"Java::mongodb.jdbc.MongoDriver", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_a16f09fa-dc00-4e9b-9c72-6da4159206d7", enable_metric=>true, charset=>"UTF-8">, jdbc_paging_enabled=>false, jdbc_page_size=>100000, jdbc_validate_connection=>false, jdbc_validation_timeout=>3600, jdbc_pool_timeout=>5, sql_log_level=>"info", connection_retry_attempts=>1, connection_retry_attempts_wait_time=>0.5, plugin_timezone=>"utc", last_run_metadata_path=>"/usr/share/logstash/.logstash_jdbc_last_run", use_column_value=>false, tracking_column_type=>"numeric", clean_run=>false, record_last_run=>true, lowercase_column_names=>true, use_prepared_statements=>false>
  Error: Java::mongodb.jdbc.MongoDriver not loaded. Are you sure you've included the correct jdbc driver in :jdbc_driver_library?

I don't think this is the correct jdbc driver

Try this one instead:
http://unityjdbc.com/download.php?type=mongodb

JDBC Driver class name:  mongodb.jdbc.MongoDriver
URL format:              jdbc:mongodb://<serverName>/<databaseName>

Also check out this community project:

i did like you said
this is the new configuration :


input {
  jdbc {
        jdbc_driver_library => "/usr/share/logstash/mongo/mongodb_unityjdbc_full.jar"
        jdbc_driver_class => "Java::mongodb.jdbc.MongoDriver"
        jdbc_connection_string => "jdbc:mongodb://<server>:27017/stg"
        jdbc_user => "<user>"
        jdbc_password => "<pass>"
        #schedule => "*/10 * * * * *"
        statement => "db.logEvents.find({});"
  }

}
output {
 file {
    path => "/var/log/logstash/mongo.log"
 }
}

and i get this error now:

<user><server> logstash]$ tail -100 logstash-plain.log  | grep -i error
[2021-12-13T09:46:25,317][ERROR][logstash.pipeline        ] A plugin had an unrecoverable error. Will restart this plugin.
  Error: unable to load /usr/share/logstash/mongojar/mongodb_unityjdbc_full.jar from :jdbc_driver_library, no such file to load -- /usr/share/logstash/mongojar/mongodb_unityjdbc_full
  Exception: LogStash::PluginLoadingError
[2021-12-13T09:46:26,332][ERROR][logstash.pipeline        ] A plugin had an unrecoverable error. Will restart this plugin.
  Error: unable to load /usr/share/logstash/mongojar/mongodb_unityjdbc_full.jar from :jdbc_driver_library, no such file to load -- /usr/share/logstash/mongojar/mongodb_unityjdbc_full
  Exception: LogStash::PluginLoadingError

It says “no such file to load” so double check your path and permissions

i made sure it is located in the path
now i get this:

[2021-12-15T10:02:15,283][DEBUG][logstash.agent           ] Converging pipelines state {:actions_count=>0}
[2021-12-15T10:02:15,984][DEBUG][logstash.inputs.jdbc     ] loading /usr/share/logstash/mongo/mongodb_unityjdbc_full.jar
[2021-12-15T10:02:15,985][ERROR][logstash.pipeline        ] A plugin had an unrecoverable error. Will restart this plugin.
  Pipeline_id:mongodb
  Plugin: <LogStash::Inputs::Jdbc jdbc_user=>"<user>", jdbc_password=><password>, statement=>"db.logEvents.find({});", jdbc_driver_library=>"/usr/share/logstash/mongo/mongodb_unityjdbc_full.jar", jdbc_connection_string=>"jdbc:mongodb://<server>:27017/stg", id=>"b26a801e0007f535ba829b03574fb36cc001fdac19bb6a42c11d81b54fd25ffe", jdbc_driver_class=>"mongodb.jdbc.MongoDriver", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_50c0bf8d-3ce0-4fe4-b5e9-74f5cadf30f1", enable_metric=>true, charset=>"UTF-8">, jdbc_paging_enabled=>false, jdbc_page_size=>100000, jdbc_validate_connection=>false, jdbc_validation_timeout=>3600, jdbc_pool_timeout=>5, sql_log_level=>"info", connection_retry_attempts=>1, connection_retry_attempts_wait_time=>0.5, plugin_timezone=>"utc", last_run_metadata_path=>"/usr/share/logstash/.logstash_jdbc_last_run", use_column_value=>false, tracking_column_type=>"numeric", clean_run=>false, record_last_run=>true, lowercase_column_names=>true, use_prepared_statements=>false>
  Error: mongodb.jdbc.MongoDriver not loaded. Are you sure you've included the correct jdbc driver in :jdbc_driver_library?
  Exception: LogStash::PluginLoadingError
  Stack: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-jdbc-4.3.19/lib/logstash/plugin_mixins/jdbc/jdbc.rb:180:in `open_jdbc_connection'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-jdbc-4.3.19/lib/logstash/plugin_mixins/jdbc/jdbc.rb:243:in `execute_statement'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-jdbc-4.3.19/lib/logstash/inputs/jdbc.rb:309:in `execute_query'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-jdbc-4.3.19/lib/logstash/inputs/jdbc.rb:281:in `run'
/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:426:in `inputworker'
/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:420:in `block in start_input'
[2021-12-15T10:02:16,160][DEBUG][logstash.outputs.file    ] Starting flush cycle

see location exist:


[clau@server mongo]$ ls
2021-10-26.zip  
drivers   
initsources.bat  
lib                               
mongodb_unityjdbc_full.jar  
ruby-3.0.3  
splunk_unityjdbc_full.jar  
unity.ico         
unityjdbc.jar
code            
icon.png  
initsources.sh   
logstash-input-mongodb-0.3.0.gem  
README.txt                  
sampleDB    
Uninstaller                
UnityJDBCDoc.pdf  
UnityJDBC_Trial_Install.jar
[clau@server mongo]$ pwd
/usr/share/logstash/mongo
[clau@server mongo]$

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.