Didn't immediately reflect or update the elasticsearch have insert last record data in table through logstash


(Karthick Tgi) #1

Using logstash 6.4.2

I have using the statement on select query for last insert the data in table.

I run with a schedule ( schedule => "* * * * *" ) for logstash instead of running,but the data is didn't update or reflect immediately in elasticsearch, once I will stop logstash reflected or updated the data in elasticsearch.

If you have any solutions?


#2

hope this helps you

this expression works fine, for every 1minute, it will take the data from the database
schedule => "*/1 * * * *"

The input code is as follows

input {

jdbc {
jdbc_driver_library => "D:/Softwares/logstash-6.4.2/lib/com.mysql.jdbc_5.1.5.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_connection_string => "jdbc:mysql://localhost:3306/test"
jdbc_user => "root"
jdbc_password => "root"
statement => "SELECT * FROM sample"
jdbc_paging_enabled => "true"
jdbc_page_size => "50000"
schedule => "*/1 * * * *"
}

}

output{
elasticsearch { codec => json hosts => ["localhost:9200"] index => "sqldata" }
stdout { codec => rubydebug }
}


(Karthick Tgi) #3

Thanks for response ,
The above method i tried is working fine, but i need last insert the data only update in database and it will reflect in elastic.
I think update data it can be stored in logstash buffer and it's take some hours it will reflected automatically in elastic. Here i need immediately reflect in elastic.. Is there any solutions for it?
I tried logstash.yml file changes lot of things it's not working..


#4

we must use pattern to update the data for every second in "schedule"


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.