# Update search results with new inserted data **URL:** \ **Category:** Logstash\ **Created:** [November 5, 2017, 5:20pm UTC](https://discuss.elastic.co/t/update-search-results-with-new-inserted-data/106437 "2017-11-05T17:20:14Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![parsa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/parsa/32/23227_2.png) [@parsa](https://discuss.elastic.co/u/parsa)\ **Post date:** [November 5, 2017, 5:20pm UTC](https://discuss.elastic.co/t/update-search-results-with-new-inserted-data/106437/1 "2017-11-05T17:20:14Z")
How can I update search results for new inserted data? What can I do for it update automatically? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [November 6, 2017, 7:10am UTC](https://discuss.elastic.co/t/update-search-results-with-new-inserted-data/106437/2 "2017-11-06T07:10:26Z")
Which search results? How is data being inserted? ---
**Author:** ![parsa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/parsa/32/23227_2.png) [@parsa](https://discuss.elastic.co/u/parsa)\ **Post date:** [November 6, 2017, 7:28am UTC](https://discuss.elastic.co/t/update-search-results-with-new-inserted-data/106437/3 "2017-11-06T07:28:30Z")
I mean new records which inserted to the table. When a new record inserts to the table, the search no have any results about that. So I think it must be there anyway for introducing new records for searching. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [November 6, 2017, 8:23am UTC](https://discuss.elastic.co/t/update-search-results-with-new-inserted-data/106437/4 "2017-11-06T08:23:51Z")
If you're talking about the jdbc input then it has a schedule option that causes the query to be run periodically. ---
**Author:** ![parsa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/parsa/32/23227_2.png) [@parsa](https://discuss.elastic.co/u/parsa)\ **Post date:** [November 6, 2017, 7:13pm UTC](https://discuss.elastic.co/t/update-search-results-with-new-inserted-data/106437/5 "2017-11-06T19:13:56Z")
``` schedule => "* * * * *" ``` The project which I work on it, is a social video share service. So you think I have a video table. For searching on the contents of this table how much time is appropriate for schedule option? And any question : How schedule option is works ? it defines new records of table on search? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [November 6, 2017, 8:54pm UTC](https://discuss.elastic.co/t/update-search-results-with-new-inserted-data/106437/6 "2017-11-06T20:54:23Z")
> For searching on the contents of this table how much time is appropriate for schedule option? How frequently does it need to be updated? How frequently is the database table updated? How expensive is the query? You'll figure something out. Worry about that once things are working. > And any question : How schedule option is works ? it defines new records of table on search? Every time the schedule fires Logstash runs the configured query and produces one Logstash event for every row in the result set. To avoid duplicates you'll want to transfer the id of the rowed of the source table into Elasticsearch so that updates in the source table result in updates in ES. By default you'll get new documents all the time. ---
**Author:** ![parsa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/parsa/32/23227_2.png) [@parsa](https://discuss.elastic.co/u/parsa)\ **Post date:** [November 8, 2017, 9:49am UTC](https://discuss.elastic.co/t/update-search-results-with-new-inserted-data/106437/7 "2017-11-08T09:49:56Z")
My video table in every minutes updates(mean : at least one record inserts to the table) and now it have about 1.000.000 records. For scheduling in every minutes, the cost is not expensive? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 8, 2017, 9:52am UTC](https://discuss.elastic.co/t/update-search-results-with-new-inserted-data/106437/8 "2017-11-08T09:52:54Z")
Why not add a trigger on the table that updates a timestamp column when it is modified? You can then compare this to the `sql_last_value` parameter to only select modified records. ---
**Author:** ![parsa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/parsa/32/23227_2.png) [@parsa](https://discuss.elastic.co/u/parsa)\ **Post date:** [November 10, 2017, 1:17pm UTC](https://discuss.elastic.co/t/update-search-results-with-new-inserted-data/106437/9 "2017-11-10T13:17:33Z")
How can I use that on schedule option? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 10, 2017, 1:27pm UTC](https://discuss.elastic.co/t/update-search-results-with-new-inserted-data/106437/10 "2017-11-10T13:27:48Z")
You write a query that just selects the rows that have a timestamp newer than when you last ran, using the `sql_last_value` parameter. You can then set it to run on an appropriate schedule, and it will only process the (relatively) few records that have changed.. ---
**Author:** ![parsa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/parsa/32/23227_2.png) [@parsa](https://discuss.elastic.co/u/parsa)\ **Post date:** [November 10, 2017, 1:51pm UTC](https://discuss.elastic.co/t/update-search-results-with-new-inserted-data/106437/11 "2017-11-10T13:51:25Z")
You mean I used that like this at my .conf file: ``` schedule => "* * * * *" statement => "SELECT * from Video where LastModifiedDate > :sql_last_value" ``` ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 10, 2017, 1:56pm UTC](https://discuss.elastic.co/t/update-search-results-with-new-inserted-data/106437/12 "2017-11-10T13:56:23Z")
Yes, something like that. ---
**Author:** ![parsa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/parsa/32/23227_2.png) [@parsa](https://discuss.elastic.co/u/parsa)\ **Post date:** [November 10, 2017, 2:02pm UTC](https://discuss.elastic.co/t/update-search-results-with-new-inserted-data/106437/13 "2017-11-10T14:02:51Z")
Thank you. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 8, 2017, 2:03pm UTC](https://discuss.elastic.co/t/update-search-results-with-new-inserted-data/106437/14 "2017-12-08T14:03:10Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.