# Bulk data import from MS SQL Server to Elasticsearch using logstash

**URL:** <https://discuss.elastic.co/t/bulk-data-import-from-ms-sql-server-to-elasticsearch-using-logstash/58731>\
**Category:** Logstash\
**Created:** [August 23, 2016, 7:12pm UTC](https://discuss.elastic.co/t/bulk-data-import-from-ms-sql-server-to-elasticsearch-using-logstash/58731 "2016-08-23T19:12:11Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![vtangutoori](https://avatars.discourse-cdn.com/v4/letter/v/ee7513/32.png) [@vtangutoori](https://discuss.elastic.co/u/vtangutoori)\
**Post date:** [August 23, 2016, 7:12pm UTC](https://discuss.elastic.co/t/bulk-data-import-from-ms-sql-server-to-elasticsearch-using-logstash/58731/1 "2016-08-23T19:12:11Z")

</div>

Hi,  
I am currently using Logstash and SQL Server JDBC to import data from SQL Server to Elasticsearch. The problem i am facing is that the SQL table has 4 million records in it and it takes forever to just upload a 100K records, is there a faster way for me to index the SQL data into Elasticsearch.

Note: I have checked river plugin and it has been removed from new version of Elasticsearch,  
I tried JDBC importer but it does not work(If anyone can give me instructions on how to use this for SQL Server i would really appreciate it.)

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 24, 2016, 5:41am UTC](https://discuss.elastic.co/t/bulk-data-import-from-ms-sql-server-to-elasticsearch-using-logstash/58731/2 "2016-08-24T05:41:09Z")

</div>

Have you identified what is limiting performance? How quickly are you able to read data from the database if you do not index them into Elasticsearch? How quickly are you able to index data into your cluster if you feed from a file instead of reading from the database?

---

<div class="post-metadata">

**Author:** ![vtangutoori](https://avatars.discourse-cdn.com/v4/letter/v/ee7513/32.png) [@vtangutoori](https://discuss.elastic.co/u/vtangutoori)\
**Post date:** [August 24, 2016, 7:51pm UTC](https://discuss.elastic.co/t/bulk-data-import-from-ms-sql-server-to-elasticsearch-using-logstash/58731/3 "2016-08-24T19:51:01Z")

</div>

Hi Christian,  
I have tried using a csv file today and even that is taking a lot of time, i have been running it for over an hour and it seems to be doing the same thing,

Regarding checking what is taking a lot of time can you please point me in the direction where i can gather information on how to do it.

Thanks  
Vamsi

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 28, 2016, 9:00am UTC](https://discuss.elastic.co/t/bulk-data-import-from-ms-sql-server-to-elasticsearch-using-logstash/58731/4 "2016-08-28T09:00:33Z")

</div>

What is the specification of your cluster/node (CPUs/RAM/Storage/Heap)? How large are you documents? What indexing throughput are you seeing? What bulk size are you using? Which version of Elasticsearch and Logstash?

---

<div class="post-metadata">

**Author:** ![chandra\_r](https://avatars.discourse-cdn.com/v4/letter/c/c2a13f/32.png) [@chandra\_r](https://discuss.elastic.co/u/chandra_r)\
**Post date:** [March 25, 2017, 2:26pm UTC](https://discuss.elastic.co/t/bulk-data-import-from-ms-sql-server-to-elasticsearch-using-logstash/58731/6 "2017-03-25T14:26:20Z")

</div>

hi vamsi,

i am on the same boat as your. i have been trying to import data in to Elastic search . i would really appreciate,if you could please shed some light on this. i had followed below link, but didn`t really work.  
[http://hintdesk.com/how-to-connect-elasticsearch-to-ms-sql-server/](http://hintdesk.com/how-to-connect-elasticsearch-to-ms-sql-server/)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:27am UTC](https://discuss.elastic.co/t/bulk-data-import-from-ms-sql-server-to-elasticsearch-using-logstash/58731/7 "2017-07-06T04:27:33Z")

</div>


