Connect Logstash to SQL by jdbc with AD Account

Oriya · July 26, 2022, 11:13am

Hi,
we have a new environment with 1 logstash server and 3 node servers.
in the logstash i have just only one pipeline that i configured with jdbc to connect to sql server.
this is the Input configuration pipeline :

input {

 jdbc {
    jdbc_driver_library => "/usr/lib/jdbc-lib/mssql-jdbc-8.2.2.jre8.jar"
    jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
    jdbc_connection_string => "jdbc:sqlserver://******\instance01;databasename=*****;"
    jdbc_user => "*******"
    jdbc_password => "********"
    use_column_value => true
    tracking_column => "id"
    tracking_column_type => "numeric"
    last_run_metadata_path => "/etc/logstash/last_run/*****_last_run.yml"
    schedule => "* * * * *"
	  statement => "s***************** where id > :sql_last_value"
    tags => ["********"]
  } 
  
 }

the user name is a AD account and i success to login with SSMS with that user to the sql server and to the database.

in the logstash server i get an error :

[ERROR][logstash.inputs.jdbc     ][main][0d6fc2b32a734ea69d00291bc666b760e63d4554dc5c4e517e833bb5534885a1]
com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user '***********'. ClientConnectionId:b8e444fd-4644-4020-bbdf-13892173773f
        at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(com/microsoft/sqlserver/jdbc/SQLServerException.java:262) ~[mssql-jdbc-8.2.2.jre8.jar:?]
        at com.microsoft.sqlserver.jdbc.TDSTokenHandler.onEOF(com/microsoft/sqlserver/jdbc/tdsparser.java:283) ~[mssql-jdbc-8.2.2.jre8.jar:?]
        at com.microsoft.sqlserver.jdbc.TDSParser.parse(com/microsoft/sqlserver/jdbc/tdsparser.java:129) ~[mssql-jdbc-8.2.2.jre8.jar:?]
        at com.microsoft.sqlserver.jdbc.TDSParser.parse(com/microsoft/sqlserver/jdbc/tdsparser.java:37) ~[mssql-jdbc-8.2.2.jre8.jar:?]
        at com.microsoft.sqlserver.jdbc.SQLServerConnection.sendLogon(com/microsoft/sqlserver/jdbc/SQLServerConnection.java:5173) ~[mssql-jdbc-8.2.2.jre8.jar:?]
        at com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(com/microsoft/sqlserver/jdbc/SQLServerConnection.java:3810) ~[mssql-jdbc-8.2.2.jre8.jar:?]
        at com.microsoft.sqlserver.jdbc.SQLServerConnection.access$000(com/microsoft/sqlserver/jdbc/SQLServerConnection.java:94) ~[mssql-jdbc-8.2.2.jre8.jar:?]
        at com.microsoft.sqlserver.jdbc.SQLServerConnection$LogonCommand.doExecute(com/microsoft/sqlserver/jdbc/SQLServerConnection.java:3754) ~[mssql-jdbc-8.2.2.jre8.jar:?]
        at com.microsoft.sqlserver.jdbc.TDSCommand.execute(com/microsoft/sqlserver/jdbc/IOBuffer.java:7225) ~[mssql-jdbc-8.2.2.jre8.jar:?]
        at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(com/microsoft/sqlserver/jdbc/SQLServerConnection.java:3053) ~[mssql-jdbc-8.2.2.jre8.jar:?]
        at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(com/microsoft/sqlserver/jdbc/SQLServerConnection.java:2562) ~[mssql-jdbc-8.2.2.jre8.jar:?]
        at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(com/microsoft/sqlserver/jdbc/SQLServerConnection.java:2216) ~[mssql-jdbc-8.2.2.jre8.jar:?]
        at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(com/microsoft/sqlserver/jdbc/SQLServerConnection.java:2067) ~[mssql-jdbc-8.2.2.jre8.jar:?]
        at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(com/microsoft/sqlserver/jdbc/SQLServerConnection.java:1204) ~[mssql-jdbc-8.2.2.jre8.jar:?]
        at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(com/microsoft/sqlserver/jdbc/SQLServerDriver.java:825) ~[mssql-jdbc-8.2.2.jre8.jar:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(jdk/internal/reflect/NativeMethodAccessorImpl.java:62) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(jdk/internal/reflect/DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:566) ~[?:?]
        at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(org/jruby/javasupport/JavaMethod.java:426) ~[jruby.jar:?]
        at org.jruby.javasupport.JavaMethod.invokeDirect(org/jruby/javasupport/JavaMethod.java:293) ~[jruby.jar:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_53_dot_0.lib.sequel.adapters.jdbc.connect(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.53.0/lib/sequel/adapters/jdbc.rb:237) ~[?:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_53_dot_0.lib.sequel.connection_pool.make_new(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.53.0/lib/sequel/connection_pool.rb:122) ~[?:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_53_dot_0.lib.sequel.connection_pool.threaded.assign_connection(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.53.0/lib/sequel/connection_pool/threaded.rb:209) ~[?:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_53_dot_0.lib.sequel.connection_pool.threaded.acquire(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.53.0/lib/sequel/connection_pool/threaded.rb:139) ~[?:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_53_dot_0.lib.sequel.connection_pool.threaded.hold(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.53.0/lib/sequel/connection_pool/threaded.rb:91) ~[?:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_53_dot_0.lib.sequel.database.connecting.synchronize(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.53.0/lib/sequel/database/connecting.rb:269) ~[?:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_53_dot_0.lib.sequel.database.connecting.test_connection(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.53.0/lib/sequel/database/connecting.rb:278) ~[?:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_53_dot_0.lib.sequel.database.misc.initialize(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.53.0/lib/sequel/database/misc.rb:175) ~[?:?]
        at org.jruby.RubyClass.newInstance(org/jruby/RubyClass.java:939) ~[jruby.jar:?]
        at org.jruby.RubyClass$INVOKER$i$newInstance.call(org/jruby/RubyClass$INVOKER$i$newInstance.gen) ~[jruby.jar:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_53_dot_0.lib.sequel.database.connecting.connect(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.53.0/lib/sequel/database/connecting.rb:57) ~[?:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.sequel_minus_5_dot_53_dot_0.lib.sequel.core.connect(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.53.0/lib/sequel/core.rb:124) ~[?:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_jdbc_minus_5_dot_2_dot_4.lib.logstash.plugin_mixins.jdbc.jdbc.jdbc_connect(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.2.4/lib/logstash/plugin_mixins/jdbc/jdbc.rb:117) ~[?:?]
        at org.jruby.RubyKernel.loop(org/jruby/RubyKernel.java:1442) ~[jruby.jar:?]
        at org.jruby.RubyKernel$INVOKER$s$0$0$loop.call(org/jruby/RubyKernel$INVOKER$s$0$0$loop.gen) ~[jruby.jar:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_jdbc_minus_5_dot_2_dot_4.lib.logstash.plugin_mixins.jdbc.jdbc.jdbc_connect(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.2.4/lib/logstash/plugin_mixins/jdbc/jdbc.rb:114) ~[?:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_jdbc_minus_5_dot_2_dot_4.lib.logstash.plugin_mixins.jdbc.jdbc.open_jdbc_connection(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.2.4/lib/logstash/plugin_mixins/jdbc/jdbc.rb:157) ~[?:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_jdbc_minus_5_dot_2_dot_4.lib.logstash.plugin_mixins.jdbc.jdbc.execute_statement(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.2.4/lib/logstash/plugin_mixins/jdbc/jdbc.rb:214) ~[?:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_jdbc_minus_5_dot_2_dot_4.lib.logstash.inputs.jdbc.execute_query(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.2.4/lib/logstash/inputs/jdbc.rb:334) ~[?:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_jdbc_minus_5_dot_2_dot_4.lib.logstash.inputs.jdbc.run(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.2.4/lib/logstash/inputs/jdbc.rb:299) ~[?:?]
        at org.jruby.RubyProc.call(org/jruby/RubyProc.java:275) ~[jruby.jar:?]
        at org.jruby.RubyProc$INVOKER$i$call.call(org/jruby/RubyProc$INVOKER$i$call.gen) ~[jruby.jar:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.rufus_minus_scheduler_minus_3_dot_0_dot_9.lib.rufus.scheduler.jobs.do_call(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/jobs.rb:234) ~[?:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.rufus_minus_scheduler_minus_3_dot_0_dot_9.lib.rufus.scheduler.jobs.do_trigger(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/jobs.rb:258) ~[?:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.rufus_minus_scheduler_minus_3_dot_0_dot_9.lib.rufus.scheduler.jobs.start_work_thread(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/jobs.rb:300) ~[?:?]
        at org.jruby.RubyProc.call(org/jruby/RubyProc.java:275) ~[jruby.jar:?]
        at org.jruby.RubyProc$INVOKER$i$call.call(org/jruby/RubyProc$INVOKER$i$call.gen) ~[jruby.jar:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.rufus_minus_scheduler_minus_3_dot_0_dot_9.lib.rufus.scheduler.jobs.start_work_thread(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/jobs.rb:299) ~[?:?]
        at org.jruby.RubyKernel.loop(org/jruby/RubyKernel.java:1442) ~[jruby.jar:?]
        at org.jruby.RubyKernel$INVOKER$s$0$0$loop.call(org/jruby/RubyKernel$INVOKER$s$0$0$loop.gen) ~[jruby.jar:?]
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.rufus_minus_scheduler_minus_3_dot_0_dot_9.lib.rufus.scheduler.jobs.start_work_thread(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/jobs.rb:289) ~[?:?]
        at org.jruby.RubyProc.call(org/jruby/RubyProc.java:318) ~[jruby.jar:?]
        at java.lang.Thread.run(java/lang/Thread.java:829) [?:?]

there is an option to connect with AD account by jdbc ?
am i using the correct jdbc library ?

thanks!

Oriya · July 26, 2022, 12:01pm

i found my problem but not the answer.

in the log i found that :

This driver is not configured for integrated authentication.

so, what do i need to do to connect via jdbc with AD account?

Ugo_Sangiorgi · July 26, 2022, 9:33pm

The problem might be that you are not specifying the authentication scheme in your jdbc_connection_string

The following was tested with the mssql-jdbc-9.4.0.jre8.jar yours is a little bit older, but try it:

jdbc_connection_string => "jdbc:sqlserver://******\instance01;databasename=*****;authenticationScheme=NTLM;integratedSecurity=true"

Oriya · July 27, 2022, 2:06pm

it's worked! @Ugo_Sangiorgi
Thank you very much for your help

5k_pwc · July 27, 2022, 6:34pm

try this code:

input {
  jdbc {
	jdbc_default_timezone => "America/New_York"
    jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
    jdbc_connection_string => "jdbc:sqlserver://localhost;databaseName=***;user=***;password=****;"
    jdbc_user => "***"
    jdbc_password => "*****"
    use_column_value => true
	tracking_column_type => "timestamp"
tracking_column => "rml_lastupdate"
	last_run_metadata_path => "C:/logstash-8.2.0/config/.Mrpres_rml_lastupdate"
	statement => "******"
 
	schedule => "*/5 * * * * *"
 }
}
output {
 elasticsearch {
	hosts => ["localhost:9200"]
	index => "yourbd"
	document_id => "%{yourID}"
 }

system · August 24, 2022, 6:35pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Logstash mssql JDBC connector Logstash	1	308	November 9, 2020
Logstash pipeline MS SQL Server Query Logstash	2	646	March 3, 2020
Logstash JDBC to MSSQL - this driver is not configured for integrated authentication Logstash	3	2422	April 6, 2020
JDBC Connection from Linux to SQL Server db Logstash	1	1468	July 6, 2017
Jdbc connection to MS-SQL server with windows credentials Logstash	2	1941	March 19, 2021

Connect Logstash to SQL by jdbc with AD Account

Related topics