Grab data from mysql using more than one tracking column

Elastic Stack Logstash
1

Hi Community

I grab data from a mysql database using logstash' on a regular basis.

In the mysql database the primary key is based on three columns instead of a single id column as primary key.

How can I set more than one key as the tracking value in the logstash pipeline? Is this possible?

Below you see "tracking_column => "id" but I would like to track based on three values: "tracking_column => ["id1", "id2", "id3"]

input {
    jdbc {
        jdbc_driver_library => "/usr/share/java/mysql-connector-java-8.0.17/mysql-connector-java-8.0.17.jar"
        jdbc_driver_class => "com.mysql.jdbc.Driver"
        jdbc_connection_string => "   "
        jdbc_user => "   "
        jdbc_password => "   "
        statement => "SELECT ..."
        use_column_value => true
        tracking_column => "id"
        jdbc_paging_enabled => true
        jdbc_page_size => 10000
        jdbc_fetch_size => 10000
        schedule => "*/1 * * * *" #every 1m
        clean_run => false
    }
}
2

If your DB has a syntax for CONCAT ... AS then you should be able to concatenate the three columns and use the synthetic result as a tracking column.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.