# How to Capture the data from a DB server using a query and visualise in kibana **URL:** \ **Category:** Logstash\ **Created:** [January 28, 2016, 10:41am UTC](https://discuss.elastic.co/t/how-to-capture-the-data-from-a-db-server-using-a-query-and-visualise-in-kibana/40349 "2016-01-28T10:41:09Z")\ **Posts on this page:** 18\ **Page:** 1
**Author:** ![chaitanyap](https://avatars.discourse-cdn.com/v4/letter/c/e5b9ba/32.png) [@chaitanyap](https://discuss.elastic.co/u/chaitanyap)\ **Post date:** [January 28, 2016, 10:41am UTC](https://discuss.elastic.co/t/how-to-capture-the-data-from-a-db-server-using-a-query-and-visualise-in-kibana/40349/1 "2016-01-28T10:41:09Z")
Hi All, We have requirement to capture the data from a remote DB server using a DB query/table and visualise that metrics into dash board. Is there any step by step procedure available to get it done? Thanks, Chaitanya. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [January 28, 2016, 6:23pm UTC](https://discuss.elastic.co/t/how-to-capture-the-data-from-a-db-server-using-a-query-and-visualise-in-kibana/40349/2 "2016-01-28T18:23:39Z")
Have a look at [https://www.elastic.co/blog/logstash-jdbc-input-plugin](https://www.elastic.co/blog/logstash-jdbc-input-plugin). ---
**Author:** ![chaitanyap](https://avatars.discourse-cdn.com/v4/letter/c/e5b9ba/32.png) [@chaitanyap](https://discuss.elastic.co/u/chaitanyap)\ **Post date:** [January 29, 2016, 4:29am UTC](https://discuss.elastic.co/t/how-to-capture-the-data-from-a-db-server-using-a-query-and-visualise-in-kibana/40349/3 "2016-01-29T04:29:56Z")
Hi Magnus, Thanks for sharing. I have installed logstash-jdbc-input-plugins. But i don't see any files created like simple-out.conf and contacts-index-logstash.conf. Is there any specific location i can find them and alter the settings. And also see that its required to run the logstash with "simple-out.conf". Will that impact the data that i am getting from beats, because in logstash config file we have given the beats as input. Please advise. Thanks for you help, Chaitanya. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [January 29, 2016, 6:40am UTC](https://discuss.elastic.co/t/how-to-capture-the-data-from-a-db-server-using-a-query-and-visualise-in-kibana/40349/4 "2016-01-29T06:40:03Z")
> But i don't see any files created like simple-out.conf and contacts-index-logstash.conf. Is there any specific location i can find them and alter the settings. You are expected to create those files yourself. > And also see that its required to run the logstash with "simple-out.conf". Will that impact the data that i am getting from beats, because in logstash config file we have given the beats as input. If you use simple-out.conf in the same Logstash instance as the beats input you'll most likely have to make adjustments to at least one of the files. Get things working when running in a separate instance first, then worry about doing it all in one instance (if that's even a requirement). ---
**Author:** ![chaitanyap](https://avatars.discourse-cdn.com/v4/letter/c/e5b9ba/32.png) [@chaitanyap](https://discuss.elastic.co/u/chaitanyap)\ **Post date:** [January 29, 2016, 6:56am UTC](https://discuss.elastic.co/t/how-to-capture-the-data-from-a-db-server-using-a-query-and-visualise-in-kibana/40349/5 "2016-01-29T06:56:17Z")
Thanks for the guidance. So you want me to install the logstash in separate server and integrate with the same kibana what we are using for beats data. Will that work? Thanks, Chaitanya. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [January 29, 2016, 6:57am UTC](https://discuss.elastic.co/t/how-to-capture-the-data-from-a-db-server-using-a-query-and-visualise-in-kibana/40349/6 "2016-01-29T06:57:04Z")
Yes, but you don't have to install it in a different server. Just run a second instance on the same server. ---
**Author:** ![chaitanyap](https://avatars.discourse-cdn.com/v4/letter/c/e5b9ba/32.png) [@chaitanyap](https://discuss.elastic.co/u/chaitanyap)\ **Post date:** [January 29, 2016, 7:01am UTC](https://discuss.elastic.co/t/how-to-capture-the-data-from-a-db-server-using-a-query-and-visualise-in-kibana/40349/7 "2016-01-29T07:01:37Z")
Ohh.. That's interesting. I will take a look into that. Thank You very Much for the help. ---
**Author:** ![chaitanyap](https://avatars.discourse-cdn.com/v4/letter/c/e5b9ba/32.png) [@chaitanyap](https://discuss.elastic.co/u/chaitanyap)\ **Post date:** [February 15, 2016, 11:24am UTC](https://discuss.elastic.co/t/how-to-capture-the-data-from-a-db-server-using-a-query-and-visualise-in-kibana/40349/8 "2016-02-15T11:24:38Z")
I have created required file for this requirement and started the new logstash instance. but it is giving below drivers error. is there any specific location that i need to copy these drivers. ```auto PS C:\logstash_DB\bin> .\logstash -f .\simple-out.conf io/console not supported; tty will not be manipulated Settings: Default filter workers: 2 Error: sun.jdbc.odbc.JdbcOdbcDriver not loaded. Are you sure you've included the correct jdbc driver in :jdbc_driver_lib rary? You may be interested in the '--configtest' flag which you can use to validate logstash's configuration before you choose to restart a running system. ``` This is the configuration i used ```auto # The path to our downloaded jdbc driver jdbc_driver_library => "C:\logstash_DB\sqljdbc_6.0\jdbc\ojdbc5.jar" # The name of the driver class for Postgresql jdbc_driver_class => "sun.jdbc.odbc.JdbcOdbcDriver" ``` Thanks, Chaitanya. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [February 15, 2016, 12:19pm UTC](https://discuss.elastic.co/t/how-to-capture-the-data-from-a-db-server-using-a-query-and-visualise-in-kibana/40349/9 "2016-02-15T12:19:07Z")
I assume you've double-checked that C:\logstash\_DB\sqljdbc\_6.0\jdbc\ojdbc5.jar exists and is readable to the user running Logstash? I'd try using forward slashes in the path instead of backslashes. ---
**Author:** ![chaitanyap](https://avatars.discourse-cdn.com/v4/letter/c/e5b9ba/32.png) [@chaitanyap](https://discuss.elastic.co/u/chaitanyap)\ **Post date:** [February 16, 2016, 7:37am UTC](https://discuss.elastic.co/t/how-to-capture-the-data-from-a-db-server-using-a-query-and-visualise-in-kibana/40349/10 "2016-02-16T07:37:52Z")
I am able to fix the issue with the drivers, I need to mention Java:: before to the driver. The query is giving the results and able to see in Kibana but it is not updating, its only showing one record since i restart the logstash DB instance. How can i make this to update for every 10 mins ? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [February 16, 2016, 8:20am UTC](https://discuss.elastic.co/t/how-to-capture-the-data-from-a-db-server-using-a-query-and-visualise-in-kibana/40349/11 "2016-02-16T08:20:18Z")
The jdbc input documentation contains a section on how to run the query periodically. ---
**Author:** ![chaitanyap](https://avatars.discourse-cdn.com/v4/letter/c/e5b9ba/32.png) [@chaitanyap](https://discuss.elastic.co/u/chaitanyap)\ **Post date:** [February 16, 2016, 9:49am UTC](https://discuss.elastic.co/t/how-to-capture-the-data-from-a-db-server-using-a-query-and-visualise-in-kibana/40349/12 "2016-02-16T09:49:30Z")
Yes, i have given it for every minute. `schedule => "* * * * *"` I am using below command to start it `.\logstash -f .\simple-out.conf` As soon as I close the power shell or terminate the command its not recording. If I leave the PS open its recording the log. Is that how it suppose to be? I need some automatic way to get the data, like how we get it from filebeat or topbeat. Thanks. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [February 16, 2016, 9:50am UTC](https://discuss.elastic.co/t/how-to-capture-the-data-from-a-db-server-using-a-query-and-visualise-in-kibana/40349/13 "2016-02-16T09:50:47Z")
Well, the schedule will only run if Logstash is running. ---
**Author:** ![chaitanyap](https://avatars.discourse-cdn.com/v4/letter/c/e5b9ba/32.png) [@chaitanyap](https://discuss.elastic.co/u/chaitanyap)\ **Post date:** [February 16, 2016, 9:52am UTC](https://discuss.elastic.co/t/how-to-capture-the-data-from-a-db-server-using-a-query-and-visualise-in-kibana/40349/14 "2016-02-16T09:52:56Z")
But I still see the logstash service is running without any issues. is there any other way to automate this? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [February 16, 2016, 10:01am UTC](https://discuss.elastic.co/t/how-to-capture-the-data-from-a-db-server-using-a-query-and-visualise-in-kibana/40349/15 "2016-02-16T10:01:05Z")
Sorry, I have no idea what you're asking. You say it works if Logstash when starting it with `.\logstash -f .\simple-out.conf`. So... what's the problem? ---
**Author:** ![chaitanyap](https://avatars.discourse-cdn.com/v4/letter/c/e5b9ba/32.png) [@chaitanyap](https://discuss.elastic.co/u/chaitanyap)\ **Post date:** [February 16, 2016, 10:18am UTC](https://discuss.elastic.co/t/how-to-capture-the-data-from-a-db-server-using-a-query-and-visualise-in-kibana/40349/16 "2016-02-16T10:18:11Z")
I am running this command Pwersell window. Logastash is sending the data if I keep open the power sell after executing this command. As soon as I tried to terminate or if I close the PS, it not recording anything. Even i tried giving N for `Terminate batch job (Y/N)?` but still no use. ```auto PS C:\logstash_DB\bin> .\logstash agent -f .\simple-out.conf io/console not supported; tty will not be manipulated Settings: Default filter workers: 2 Logstash startup completed { "count(1)" => 13, "@version" => "1", "@timestamp" => "2016-02-16T10:08:03.330Z" } ←[33mSIGINT received. Shutting down the pipeline. {:level=>:warn}←[0m ^CTerminate batch job (Y/N)? Logstash shutdown completed n PS C:\logstash_DB\bin> PS C:\logstash_DB\bin> ``` So how can keep the logstash to run always and execute this query for every minute ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [February 16, 2016, 11:54am UTC](https://discuss.elastic.co/t/how-to-capture-the-data-from-a-db-server-using-a-query-and-visualise-in-kibana/40349/17 "2016-02-16T11:54:48Z")
You need to run Logstash as a service. I'm not very familiar with that myself and I don't think there's an official way of doing it, but googling the topic results in lots of hits. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 5:11am UTC](https://discuss.elastic.co/t/how-to-capture-the-data-from-a-db-server-using-a-query-and-visualise-in-kibana/40349/18 "2017-07-06T05:11:13Z")