# Auto Update logs without running logstash **URL:** \ **Category:** Logstash\ **Created:** [October 23, 2017, 7:56am UTC](https://discuss.elastic.co/t/auto-update-logs-without-running-logstash/104855 "2017-10-23T07:56:28Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![Samreen\_Zeeshan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samreen_zeeshan/32/22264_2.png) [@Samreen\_Zeeshan](https://discuss.elastic.co/u/Samreen_Zeeshan)\ **Post date:** [October 23, 2017, 7:56am UTC](https://discuss.elastic.co/t/auto-update-logs-without-running-logstash/104855/1 "2017-10-23T07:56:29Z")
I have logs stored in a MS SQL table and i have migrated all logs by running logstash once. Now what i want is if i update a particular record in SQL table that should be reflected in Kibana visualizations without restarting kibana or logstash, just by turning auto refresh on. Is it possible if yes how can i do that. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [October 25, 2017, 3:35am UTC](https://discuss.elastic.co/t/auto-update-logs-without-running-logstash/104855/2 "2017-10-25T03:35:12Z")
It's not possible using Logstash at this stage sorry, it only looks for new data that has been added. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [October 25, 2017, 5:22am UTC](https://discuss.elastic.co/t/auto-update-logs-without-running-logstash/104855/3 "2017-10-25T05:22:18Z")
If you have a "last modified" column in the database you can use that to identify rows that have changed since the last run. Combined with the jdbc input's schedule feature you can get near-realtime updates of the ES index. ---
**Author:** ![Samreen\_Zeeshan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samreen_zeeshan/32/22264_2.png) [@Samreen\_Zeeshan](https://discuss.elastic.co/u/Samreen_Zeeshan)\ **Post date:** [October 30, 2017, 7:58am UTC](https://discuss.elastic.co/t/auto-update-logs-without-running-logstash/104855/4 "2017-10-30T07:58:49Z")
Added scheduler in logstash config file but still having the same problem!!! index is not modifying automatically i have to delete it and recreate it then i get desired no. of docs here is my config file. input { jdbc { jdbc\_driver\_library =\> "C:\folder\Java\jre1.8.0\_144\bin\sqljdbc42.jar" jdbc\_driver\_class =\> "com.microsoft.sqlserver.jdbc.SQLServerDriver" jdbc\_connection\_string =\> "jdbc:sqlserver://myserver" jdbc\_user =\> "username" jdbc\_password =\> "password" schedule =\> "\* \* \* \* \*" statement =\> "SELECT \* FROM Logs where Status != 2 } } output { elasticsearch { hosts =\> ["localhost:9200"] index =\> "index\_name" document\_type =\> "log" document\_id =\> "%{log\_oid}" ``` } ``` } ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [October 30, 2017, 8:04am UTC](https://discuss.elastic.co/t/auto-update-logs-without-running-logstash/104855/5 "2017-10-30T08:04:31Z")
> Still having the same problem!!! And the answer I gave you still applies. ---
**Author:** ![Samreen\_Zeeshan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samreen_zeeshan/32/22264_2.png) [@Samreen\_Zeeshan](https://discuss.elastic.co/u/Samreen_Zeeshan)\ **Post date:** [October 30, 2017, 8:11am UTC](https://discuss.elastic.co/t/auto-update-logs-without-running-logstash/104855/6 "2017-10-30T08:11:16Z")
I have a column named "Staus" in DB table and I am only fetching records which are not resolved, plus I have added jdbc's schedule as well it runs query every minute. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [October 30, 2017, 8:19am UTC](https://discuss.elastic.co/t/auto-update-logs-without-running-logstash/104855/7 "2017-10-30T08:19:21Z")
It sounds like Logstash has no way of knowing what has changed in the last minute so you'll either have to pull all rows every minute or implement some monotonically increasing column value (like a "last modified" timestamp) that Logstash can use to pull only unprocessed rows. ---
**Author:** ![Samreen\_Zeeshan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samreen_zeeshan/32/22264_2.png) [@Samreen\_Zeeshan](https://discuss.elastic.co/u/Samreen_Zeeshan)\ **Post date:** [October 30, 2017, 8:27am UTC](https://discuss.elastic.co/t/auto-update-logs-without-running-logstash/104855/8 "2017-10-30T08:27:00Z")
I am actually fetching all rows see query SELECT \* FROM Logs where Status != 2 if i change status for some rows they are still at index ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [October 30, 2017, 9:28am UTC](https://discuss.elastic.co/t/auto-update-logs-without-running-logstash/104855/9 "2017-10-30T09:28:41Z")
The jdbc input fetches existing rows. It's not able discover what rows have been removed from the result set and therefore should be deleted from ES. That kind of synchronization requires some extra work. ---
**Author:** ![Samreen\_Zeeshan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samreen_zeeshan/32/22264_2.png) [@Samreen\_Zeeshan](https://discuss.elastic.co/u/Samreen_Zeeshan)\ **Post date:** [October 30, 2017, 11:31am UTC](https://discuss.elastic.co/t/auto-update-logs-without-running-logstash/104855/10 "2017-10-30T11:31:09Z")
like if I update Status = 2 of some rows and have scheduled logstash to run below query SELECT \* FROM Logs where Status != 2 logstash executes query but in kibana i don't get it reflected ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 27, 2017, 11:31am UTC](https://discuss.elastic.co/t/auto-update-logs-without-running-logstash/104855/11 "2017-11-27T11:31:11Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.