How can I update search results for new inserted data?
What can I do for it update automatically?
Which search results? How is data being inserted?
I mean new records which inserted to the table.
When a new record inserts to the table, the search no have any results about that.
So I think it must be there anyway for introducing new records for searching.
If you're talking about the jdbc input then it has a schedule option that causes the query to be run periodically.
schedule => "* * * * *"
The project which I work on it, is a social video share service.
So you think I have a video table.
For searching on the contents of this table how much time is appropriate for schedule option?
And any question : How schedule option is works ? it defines new records of table on search?
For searching on the contents of this table how much time is appropriate for schedule option?
How frequently does it need to be updated? How frequently is the database table updated? How expensive is the query? You'll figure something out. Worry about that once things are working.
And any question : How schedule option is works ? it defines new records of table on search?
Every time the schedule fires Logstash runs the configured query and produces one Logstash event for every row in the result set. To avoid duplicates you'll want to transfer the id of the rowed of the source table into Elasticsearch so that updates in the source table result in updates in ES. By default you'll get new documents all the time.
My video table in every minutes updates(mean : at least one record inserts to the table) and now it have about 1.000.000 records.
For scheduling in every minutes, the cost is not expensive?
Why not add a trigger on the table that updates a timestamp column when it is modified? You can then compare this to the sql_last_value parameter to only select modified records.
1 Like
How can I use that on schedule option?
You write a query that just selects the rows that have a timestamp newer than when you last ran, using the sql_last_value parameter. You can then set it to run on an appropriate schedule, and it will only process the (relatively) few records that have changed..
1 Like
You mean I used that like this at my .conf file:
schedule => "* * * * *"
statement => "SELECT * from Video where LastModifiedDate > :sql_last_value"Yes, something like that.
Thank you.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.