Synchronize data to es and customize es index structure


(王奕然) #1

i have two table: customer,customer_order customer have customer_name,sex,customerid columns. and i want synchronize the two table to elasticsearch customerOrder index the customerOrder structure like

{
  orderid:x,
  ordercreatetime,
  customerinfo:{
    customername:xx,
    sex:x
    customerid:xx
  }
}

my question is ,how to make two table data to synchronize and integration to customed es index structure like: customerOrder and when customer_name changed in customer table,i want to change all customerOrder index's customername field value. thanks all your suggestion!


#2

use below input code to connect to databse and load data from both the tables,

in case of "statement", write join query to retrieve data from both tables,

input {

jdbc {
jdbc_driver_library => "D:/Softwares/logstash-6.4.2/lib/com.mysql.jdbc_5.1.5.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_connection_string => "jdbc:mysql://localhost:3306/test"
jdbc_user => "root"
jdbc_password => "root"
statement => "SELECT * FROM sample"
jdbc_paging_enabled => "true"
jdbc_page_size => "50000"
}

}

output{
elasticsearch { codec => json hosts => ["localhost:9200"] index => "mysqldata" }
stdout { codec => rubydebug }
}


Logstash not able to output to ES error occurred sending bulk request
(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.