Delete Sync Operation using Logstash JDBC

Elastic Stack Logstash
1

Hi all,

We have configured JDBC logstash, to migrate data from Cassandra to logstash.
we have able to move data from Cassandra to logstash, we are also updating the data in elastic search based on document_id, but we not able to delete the records that deleted in Cassandra.

Can u tell us the config to delete the records in elastic search based on deleting in Cassandra?
Ultimately we need to sync two DB's using logstash JDBC.

Our Configuration:

# emp_id -- Primary Key in Cassandra

input {
    jdbc {
        jdbc_connection_string => "jdbc:cassandra://X.X.X.X:9042/cloud"
        jdbc_user => "cassandra"
        jdbc_password => "cassandra"
        jdbc_driver_library => "/usr/share/logstash/logstash-core/lib/jars/CassandraJDBC42.jar"
        jdbc_validate_connection => true
        jdbc_driver_class => "com.simba.cassandra.jdbc42.Driver"
        statement => "SELECT * FROM cloud.employee_list"
        schedule => "* * * * *"
        use_column_value => true
        tracking_column => "emp_id"
    }

}

output {
  elasticsearch { 
   hosts => ["localhost:9200"]
   document_id=> "%{emp_id}"
   index => "employeelist_table2"
  } 
  stdout { codec => rubydebug }
}

Thanks!!

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.