# SQL Server stream into Elastic Search

**URL:** <https://discuss.elastic.co/t/sql-server-stream-into-elastic-search/46908>\
**Category:** Elasticsearch\
**Created:** [April 10, 2016, 6:48pm UTC](https://discuss.elastic.co/t/sql-server-stream-into-elastic-search/46908 "2016-04-10T18:48:22Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Wayne\_Taylor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wayne_taylor/32/45984_2.png) [@Wayne\_Taylor](https://discuss.elastic.co/u/Wayne_Taylor)\
**Post date:** [April 10, 2016, 6:48pm UTC](https://discuss.elastic.co/t/sql-server-stream-into-elastic-search/46908/1 "2016-04-10T18:48:23Z")

</div>

Sorry if this has been mentioned before

I am looking to stream in data from Microsoft SQL Server, I looked at Rivers and then found out it was deprecated, so wasted a little time.

So then I looked at what next, so now I see, use logstash. I have a simple configuration working and its importing data and scheduled to run at set intervals. Whilst this is good, its batch in nature.

Long term goal is for me to work with the data creators to insert a document into elastic when the event happens. However, in the short terms is there any solution outside of scheduled logstash jobs?

Thanks  
Wayne

---

<div class="post-metadata">

**Author:** ![JunWang](https://avatars.discourse-cdn.com/v4/letter/j/f14d63/32.png) [@JunWang](https://discuss.elastic.co/u/JunWang)\
**Post date:** [April 11, 2016, 2:49am UTC](https://discuss.elastic.co/t/sql-server-stream-into-elastic-search/46908/2 "2016-04-11T02:49:48Z")

</div>

I have been using logstash jdbc driver to run SQL query and load data into elasticsearch.  
Here is my logstash config file. Ran on Windows 7 just fine.  
Hope it helps.

input {  
jdbc {  
jdbc\_driver\_library =\> "C:\Elastic\logstash-2.2.0\sqljdbc42.jar"  
jdbc\_driver\_class =\> "com.microsoft.sqlserver.jdbc.SQLServerDriver"  
jdbc\_connection\_string =\> "jdbc:sqlserver://SQLDEVL01:1433;databaseName=AnalyticDB;user=elasticsearch;password=elasticsearch;"  
jdbc\_user =\> "elasticsearch"  
jdbc\_password =\> "elasticsearch"  
statement =\> "select \* from auth"  
}  
}  
filter {  
}  
output {  
elasticsearch {  
hosts =\> "sqlprod01"  
index =\> "auths"  
document\_type =\> "auth"  
document\_id =\> "%{authref}"  
}  
stdout { codec =\> rubydebug }  
}

---

<div class="post-metadata">

**Author:** ![Wayne\_Taylor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wayne_taylor/32/45984_2.png) [@Wayne\_Taylor](https://discuss.elastic.co/u/Wayne_Taylor)\
**Post date:** [April 12, 2016, 12:38pm UTC](https://discuss.elastic.co/t/sql-server-stream-into-elastic-search/46908/3 "2016-04-12T12:38:14Z")

</div>

Hi JunWang, I understand this bit and have it working. The question here is it just scheduling this to run or is there an option outside of the deprecated rivers to auto-run this. I guess not? Without long term having people create the documents.

---

<div class="post-metadata">

**Author:** ![JunWang](https://avatars.discourse-cdn.com/v4/letter/j/f14d63/32.png) [@JunWang](https://discuss.elastic.co/u/JunWang)\
**Post date:** [April 15, 2016, 5:36pm UTC](https://discuss.elastic.co/t/sql-server-stream-into-elastic-search/46908/4 "2016-04-15T17:36:52Z")

</div>

I think I would use JDBC driver logstash for initial data conversion. For ongoing more real time data synchronization I like to use SQL Server service broker. I am currently working on this as well but not finished yet.

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [April 15, 2016, 9:06pm UTC](https://discuss.elastic.co/t/sql-server-stream-into-elastic-search/46908/5 "2016-04-15T21:06:54Z")

</div>

You can try JDBC importer [https://github.com/jprante/elasticsearch-jdbc](https://github.com/jprante/elasticsearch-jdbc)

---

<div class="post-metadata">

**Author:** ![Wayne\_Taylor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wayne_taylor/32/45984_2.png) [@Wayne\_Taylor](https://discuss.elastic.co/u/Wayne_Taylor)\
**Post date:** [April 19, 2016, 12:34pm UTC](https://discuss.elastic.co/t/sql-server-stream-into-elastic-search/46908/6 "2016-04-19T12:34:39Z")

</div>

this might seem a strange response, but how is this different than using logstash jdbc?

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [April 20, 2016, 10:20pm UTC](https://discuss.elastic.co/t/sql-server-stream-into-elastic-search/46908/7 "2016-04-20T22:20:13Z")

</div>

It is older, written in Java, and has advanced features like auto GeoJSON and auto JSON detection, also hints for JDBC result set streaming [http://docs.oracle.com/javase/8/docs/api/java/sql/Statement.html#setFetchSize-int-](http://docs.oracle.com/javase/8/docs/api/java/sql/Statement.html#setFetchSize-int-) and other goodies like merging rows into JSON docs or automatic JDBC to JSON type conversions.

---

<div class="post-metadata">

**Author:** ![endrit-b](https://avatars.discourse-cdn.com/v4/letter/e/6f9a4e/32.png) [@endrit-b](https://discuss.elastic.co/u/endrit-b)\
**Post date:** [May 17, 2017, 11:13pm UTC](https://discuss.elastic.co/t/sql-server-stream-into-elastic-search/46908/8 "2017-05-17T23:13:19Z")

</div>

Hi @jprante,

Is there any release of ES-jdbc for ES v5.0 ? (because I coudn't find it)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:00pm UTC](https://discuss.elastic.co/t/sql-server-stream-into-elastic-search/46908/9 "2017-07-05T22:00:19Z")

</div>


