# Notify Logstash when new data is entered in mysql databse without using parameter schedule **URL:** \ **Category:** Elasticsearch\ **Created:** [May 29, 2018, 6:19am UTC](https://discuss.elastic.co/t/notify-logstash-when-new-data-is-entered-in-mysql-databse-without-using-parameter-schedule/133621 "2018-05-29T06:19:38Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![iwayankit](https://avatars.discourse-cdn.com/v4/letter/i/58956e/32.png) [@iwayankit](https://discuss.elastic.co/u/iwayankit)\ **Post date:** [May 29, 2018, 6:19am UTC](https://discuss.elastic.co/t/notify-logstash-when-new-data-is-entered-in-mysql-databse-without-using-parameter-schedule/133621/1 "2018-05-29T06:19:39Z")
I am working on Elastic Stack with Mysql. everything is working fine like logstash taking data from mysql database and sending it to elasticsearch and when new entries entered in mysql data then to update elasticsearch automatically i am using parameter: Schedule but in this case logstash is checking continuously for new data from it's terminal that is my main concern. ``` input { jdbc { jdbc_connection_string => "jdbc:mysql://localhost:3306/testdb" # The user we wish to execute our statement as jdbc_user => "root" jdbc_password => "" # The path to our downloaded jdbc driver jdbc_driver_library => "/home/Downloads/mysql-connector-java-5.1.38.jar" jdbc_driver_class => "com.mysql.jdbc.Driver" #run logstash at an interval of on minute schedule => "*/15 * * * *" use_column_value => true tracking_column => 'EVENT_TIME_OCCURRENCE_FIELD' # our query statement => "SELECT * FROM xyz" } } output { stdout { codec => json_lines } elasticsearch { "hosts" => "localhost:9200" "index" => "test-migrate" "document_type" => "data" "document_id" => "%{personid}" } } ``` But if data is large Logstash will check for new entries in entire data without any stopping point then this will reduce scalability and consume more power. Is there any other method or any webhook like when new data is entered into database then mysql will notify Logstash only for new data or Logstash will check for only new entries, Please help ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [May 29, 2018, 6:41am UTC](https://discuss.elastic.co/t/notify-logstash-when-new-data-is-entered-in-mysql-databse-without-using-parameter-schedule/133621/2 "2018-05-29T06:41:34Z")
> [@iwayankit](#): > > Is there any other method or any webhook like when new data is entered into database Then it's more a MySQL question IMO as this would happen on MySQL side before elastic stack is called. Note that here you are running: ```auto statement => "SELECT * FROM xyz" ``` There is no `WHERE` clause. If in your table you have something like a timestamp, you can then just ask for the newest changes since the last run. But I prefer to modify the application layer. I shared some thoughts at [https://david.pilato.fr/blog/2015-05-09-advanced-search-for-your-legacy-application/](http://david.pilato.fr/blog/2015/05/09/advanced-search-for-your-legacy-application/). May be that could help. ---
**Author:** ![iwayankit](https://avatars.discourse-cdn.com/v4/letter/i/58956e/32.png) [@iwayankit](https://discuss.elastic.co/u/iwayankit)\ **Post date:** [May 29, 2018, 6:47am UTC](https://discuss.elastic.co/t/notify-logstash-when-new-data-is-entered-in-mysql-databse-without-using-parameter-schedule/133621/3 "2018-05-29T06:47:10Z")
thanks for your response, i am using where clause in my query as below , statement =\> "SELECT \* FROM brainplay WHERE EVENT\_TIME\_OCCURRENCE\_FIELD \> :sql\_last\_value" but this is doing same, did you get my point? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [May 29, 2018, 6:48am UTC](https://discuss.elastic.co/t/notify-logstash-when-new-data-is-entered-in-mysql-databse-without-using-parameter-schedule/133621/4 "2018-05-29T06:48:03Z")
So you mean that ALL records are fetched again? Instead of only the ones which changed? ---
**Author:** ![iwayankit](https://avatars.discourse-cdn.com/v4/letter/i/58956e/32.png) [@iwayankit](https://discuss.elastic.co/u/iwayankit)\ **Post date:** [May 29, 2018, 6:52am UTC](https://discuss.elastic.co/t/notify-logstash-when-new-data-is-entered-in-mysql-databse-without-using-parameter-schedule/133621/5 "2018-05-29T06:52:06Z")
actually logstash is fetching data continuously and every moment it is showing all inserted data on terminal that is very costly, i am looking for that it'll fetch only new entries and update it to elasticsearch ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [May 29, 2018, 6:55am UTC](https://discuss.elastic.co/t/notify-logstash-when-new-data-is-entered-in-mysql-databse-without-using-parameter-schedule/133621/6 "2018-05-29T06:55:46Z")
Then something is wrong in your logstash configuration. Check it. I'm not sure that this is needed: ```auto use_column_value => true tracking_column => 'EVENT_TIME_OCCURRENCE_FIELD' ``` BTW printing everything in the console is also something you should only use for debugging purpose as it's slowing down everything as well. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 26, 2018, 6:57am UTC](https://discuss.elastic.co/t/notify-logstash-when-new-data-is-entered-in-mysql-databse-without-using-parameter-schedule/133621/7 "2018-06-26T06:57:01Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.