[ERROR][logstash.pipeline ] Error registering plugin {:pipeline_id=>"main", :plugin=>"<LogStash::Inputs::Jdbc jdbc_driver_library=>\"/usr/share/logstash/logstash-core/lib/jars/sqljdbc42.jar\", jdbc_connection_string

Elastic Stack Logstash
1

Hello,
I have been trying to run logstash as a service but it keeps failing then restarting in a loop. I tried looking for a solution in similar topics without finding anything useful in my case.
Here are the logs when i start it:

[ERROR][logstash.pipeline        ] Error registering plugin {:pipeline_id=>"main", :plugin=>"<LogStash::Inputs::Jdbc jdbc_driver_library=>\"/usr/share/logstash/logstash-core/lib/jars/sqljdbc42.jar\", jdbc_connection_string=>\"jdbc:sqlserver://srv_sql:1036/DB_Name\", jdbc_driver_class=>\"com.microsoft.sqlserver.jdbc.SQLServerDriver\", jdbc_user=>\"srv_sql\", jdbc_password=><password>, id=>\"b6ae1db78c69d6c71246bb6df76442e27565d758bb0e40fa7632b1d2dcdb8c10\", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>\"plain_a7aa2e16-ef44-4d44-bb5f-cb51a81df76b\", enable_metric=>true, charset=>\"UTF-8\">, jdbc_paging_enabled=>false, jdbc_page_size=>100000, jdbc_validate_connection=>false, jdbc_validation_timeout=>3600, jdbc_pool_timeout=>5, sql_log_level=>\"info\", connection_retry_attempts=>1, connection_retry_attempts_wait_time=>0.5, plugin_timezone=>\"utc\", last_run_metadata_path=>\"/usr/share/logstash/.logstash_jdbc_last_run\", use_column_value=>false, tracking_column_type=>\"numeric\", clean_run=>false, record_last_run=>true, lowercase_column_names=>true, use_prepared_statements=>false>", :error=>"Must set either :statement or :statement_filepath. Only one may be set at a time.", :thread=>"#<Thread:0x71a4931e@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:245 run>"}
[2020-10-19T16:13:35,039][ERROR][logstash.pipeline        ] Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<LogStash::ConfigurationError: Must set either :statement or :statement_filepath. Only one may be set at a time.>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-jdbc-4.3.19/lib/logstash/inputs/jdbc.rb:227:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:343:in `register_plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:354:in `block in register_plugins'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:354:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:510:in `start_inputs'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:401:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:288:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:248:in `block in start'"], :thread=>"#<Thread:0x71a4931e@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:245 run>"}
[2020-10-19T16:13:35,052][ERROR][logstash.agent           ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: LogStash::PipelineAction::Create/pipeline_id:main, action_result: false", :backtrace=>nil}

I understand the issue comes from the jdbc plugin, but I don't know why... I tried removing the plug in, but a different error comes up:

[2020-10-19T16:08:17,381][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::PluginLoadingError", :message=>"Couldn't find any input plugin named 'jdbc'. Are you sure this is correct? Trying to load the jdbc input plugin resulted in this error: no such file to load -- logstash/inputs/jdbc", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/plugins/registry.rb:192:in `lookup_pipeline_plugin'", "/usr/share/logstash/logstash-core/lib/logstash/plugin.rb:140:in `lookup'", "/usr/share/logstash/logstash-core/lib/logstash/plugins/plugin_factory.rb:82:in `plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:114:in `plugin'", "(eval):16:in `<eval>'", "org/jruby/RubyKernel.java:994:in `eval'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:86:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:171:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:40:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:335:in `block in converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:332:in `block in converge_state'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:319:in `converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:166:in `block in converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:164:in `converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:90:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:343:in `block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}

Thank you for your help.

2

Did you download the correct driver and put it in correctly specified jdbc_driver_library ?

3

Hi kavierkoo, thank you for your answer

This is the driver I installed:
/usr/share/logstash/bin/logstash-plugin install logstash-input-jdbc Installing logstash-input-jdbc which: no javac in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/home/kroot/.local/bin:/home/kroot/bin) Installation successful

As for the jdbc driver library, here is the line you're asking for in the conf file (/etc/logstash/conf.d/mssql_jdbc.conf):

jdbc_driver_library => "/usr/share/logstash/logstash-core/lib/jars/sqljdbc42.jar"

4

after you installed, running logstash-plugin list is logstash-input-jdbc in the list?
I'm thinking you hit the issue even you installed and it shows successful but it did not actually installed it logstash-input-jdbc

5

The command /usr/share/logstash/bin/logstash-plugin list --verbose | grep jdbc returns the following result:

logstash-filter-jdbc_streaming (1.0.3) logstash-input-jdbc (4.3.19)
I guess it is installed, isn't it? I may need to verify elsewhere?

6

hmm, I am sorry but I'm out of clue what happened to your jdbc input.

Tho i found this, wonder will it helps your or not

Per Error: oracle.jdbc.driver.OracleDriver not loaded. · Issue #43 · logstash-plugins/logstash-input-jdbc · GitHub, prepending Java:: to the driver class may be required if it appears that the driver is not being loaded correctly despite relevant jar(s) being provided by either via the jdbc_driver_library setting or being placed in the Logstash Java classpath. This is known to be the case for the Oracle JDBC driver (ojdbc6.jar), where the correct jdbc_driver_class is "Java::oracle.jdbc.driver.OracleDriver" , and may also be the case for other JDBC drivers.

7

Alright, i'll try to resolve this
thank you for your help!

8

I tried removing the plug in then reinstalled it:

/usr/share/logstash/bin/logstash-plugin install logstash-input-jdbc
Validating logstash-input-jdbc
Installing logstash-input-jdbc
which: no javac in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/home/kroot/.local/bin:/home/kroot/bin)
Installation successful

So I checked the java version:

java -version
openjdk version "1.8.0_252"
OpenJDK Runtime Environment (build 1.8.0_252-b09)
OpenJDK 64-Bit Server VM (build 25.252-b09, mixed mode)
9

What does your jdbc input configuration look like?

10

Here it is:

input {
  jdbc {
    jdbc_driver_library => "/usr/share/logstash/logstash-core/lib/jars/sqljdbc42.jar"
    jdbc_connection_string => "jdbc:sqlserver://mssql-srv:1433/ITRS"
    jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
    jdbc_user => "mssql-user"
    jdbc_password => "password"

    statement => "SELECT * FROM Database1 
  }
}

output {
  	stdout { codec => json_lines }
	elasticsearch {
    		hosts => ["10.214.80.90:9200", "10.178.32.30:9200"]
    		index => "index1"
  }
}
11

There is a double quote missing from the end of the statement option.

1 Like
12

Thank you, that solved it!

13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.