Mysql DB sync of all tables in DB at once

[raghav1107 (Raghavendra)]

Hi,

We have mysql database which has 10 to 12 tables. Currently we are using log stash conf as below and jdbc plugin to sync the table data into elastic search. some tables has lakhs of records and might take time to sync.

Is there a way to specify all the tables of the DB in the configuration file and any updates/inserts in mysql should also sync elastic search ? Please let me know how to proceed on this.

input {
jdbc{
jdbc_connection_string => "jdbc:mysql://localhost:3306/STARWARS"
jdbc_user => "root"
jdbc_password => "starwars"
jdbc_driver_library => "/opt/jdk1.5.0_05/lib/mysql-connector-java-5.0.4-bin.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
statement => "select * from roles"
}
}
output {
stdout {codec => json_lines}
elasticsearch {
"hosts" => "localhost:9200"
"index" => "code_coverage_net1"
"user" => "elastic"
"password" => "elastic"
}
}

Thanks,
Raghav

[val (Val Crettaz)]

As explained in the documentation for the jdbc input plugin, in order to get Logstash to execute multiple statements (one per table), you need to add one jdbc input for each statement.

So simply copy/paste the one you have as many times as tables you want to read and change the statement to read from the appropriate tables.