Log stash configuration file to push delta of data from SQL Server using JDBC Importer


Any one has done this delta import in Logstatsh from sql server. my configuration file looks like this, but it fetches all the rows every time it is running. I need only the DELTA of data ( what changed/added/deleted) only to my ES.
Suggestions are appreciated.

input {
jdbc {
jdbc_connection_string => "jdbc:sqlserver://XXXXX:1433;databaseName=XXX"
jdbc_user => "XXX"
jdbc_password => "XXXX"
jdbc_validate_connection => true
jdbc_driver_library => "sqljdbc42.jar"
jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
statement => "SELECT * from XXXXXXXXXX"

output {
hosts => "localhost:9200"
index => "test"
stdout { }

Logstash won't magically know what's new and what's old. You have to change the query to only select the lines that haven't been processed. See https://www.elastic.co/guide/en/logstash/current/plugins-inputs-jdbc.html#_state and how the sql_last_value parameter is used in the example.