Data not updating automatically when I change my MySQL table

Elastic Stack Kibana
1

Hi!

I'm having some issues at the moment of updating my data in Kibana when I change any info from MySQL table.
I don't really know if it's possible.
Can someone help me?

Thanks!

2

Hello,

How are you connecting the data from your MySQL table? I am assuming you are connecting to ES and then you are trying to find that data in Kibana?
Can you see that changed data in your elasticsearch directly?

Thanks,
Bhavya

3

Hello,

I'm connecting the data from MySQL via logstash with this configuration:

input {
jdbc {
clean_run => true
jdbc_driver_library => "C:\Users\admpablos\Documents\Elastic\logstash-7.12.1\mysql-connector-java-5.1.44-bin.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_connection_string => "jdbc:mysql://localhost:3306/MY_BBDD?useSSL=false"
jdbc_user => "root"
jdbc_password => "MY_PASSWORD"
schedule => "*/5 * * * * *"
statement => "SELECT * from MY_TABLE WHERE ID > :sql_last_value"
use_column_value => true
tracking_column => "id"
}
}

output {
elasticsearch {
hosts => ["localhost:9200"]
index => "MY_INDEX"
user => "elastic"
password => "MY_PASSWORD"
}
stdout{ codec => rubydebug }
}

I guess there is something wrong with this config.
When i add any line to my MySQL table, the Data Table in Kibana is updating perfectly. The problem comes when i modify any data from any existing line of the MySQL table. This modification is not seen in the Data Table of Kibana.

Thank you for your time Bhavya!

4

Can someone help me? I don't really know if ES is able to make this function.

Thank you!

5

Hi,

I don't think LogStash is the problem here: If you update an existing entry the ID is not changed, right? But you told LogStash to store the last read entry Id so existing entries are never read again.

I think you have to change the logic to use a last modification date for this and make sure that this is changed every time the entry is updated. Then, you have to configure the elasticsearch output to use your id column as document id - otherwise a modified entry would create a new entry instead of updating the existing entry.

Best regards
Wolfram

6

Thanks you a lot Wolfram,

I'll try to make this changes. If i succed i'll post the code here.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.