# Elasticsearch and sql tables

**URL:** <https://discuss.elastic.co/t/elasticsearch-and-sql-tables/154261>\
**Category:** Elasticsearch\
**Created:** [October 26, 2018, 11:19pm UTC](https://discuss.elastic.co/t/elasticsearch-and-sql-tables/154261 "2018-10-26T23:19:29Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Garry](https://avatars.discourse-cdn.com/v4/letter/g/bb73d2/32.png) [@Garry](https://discuss.elastic.co/u/Garry)\
**Post date:** [October 26, 2018, 11:19pm UTC](https://discuss.elastic.co/t/elasticsearch-and-sql-tables/154261/1 "2018-10-26T23:19:29Z")

</div>

I have data stored in a sql table that i would like to load into elasticsearch.  
What would be the best way to do this.  
The table is updated frequently and i need elasticsearch to reflect this.

Garry

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [October 27, 2018, 12:40am UTC](https://discuss.elastic.co/t/elasticsearch-and-sql-tables/154261/2 "2018-10-27T00:40:01Z")

</div>

Logstash has a JDBC input you could use to pull data from the table periodically.

Otherwise you should consider writing to both the DB and Elasticsearch at the same time, if you want more real time updates.

---

<div class="post-metadata">

**Author:** ![Garry](https://avatars.discourse-cdn.com/v4/letter/g/bb73d2/32.png) [@Garry](https://discuss.elastic.co/u/Garry)\
**Post date:** [October 27, 2018, 5:59pm UTC](https://discuss.elastic.co/t/elasticsearch-and-sql-tables/154261/3 "2018-10-27T17:59:37Z")

</div>

What about using filebeats as it seems to be more lightweight?  
I could then export the db data to a flat file on a regular basis.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [October 28, 2018, 7:57am UTC](https://discuss.elastic.co/t/elasticsearch-and-sql-tables/154261/4 "2018-10-28T07:57:59Z")

</div>

That could work.

---

<div class="post-metadata">

**Author:** ![Garry](https://avatars.discourse-cdn.com/v4/letter/g/bb73d2/32.png) [@Garry](https://discuss.elastic.co/u/Garry)\
**Post date:** [October 28, 2018, 8:37am UTC](https://discuss.elastic.co/t/elasticsearch-and-sql-tables/154261/5 "2018-10-28T08:37:39Z")

</div>

In order to stop data duplication would i need to make the document\_id a primary key to replicate the sql table pk?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [October 28, 2018, 8:56am UTC](https://discuss.elastic.co/t/elasticsearch-and-sql-tables/154261/6 "2018-10-28T08:56:02Z")

</div>

Yep.

---

<div class="post-metadata">

**Author:** ![Garry](https://avatars.discourse-cdn.com/v4/letter/g/bb73d2/32.png) [@Garry](https://discuss.elastic.co/u/Garry)\
**Post date:** [October 28, 2018, 9:59am UTC](https://discuss.elastic.co/t/elasticsearch-and-sql-tables/154261/7 "2018-10-28T09:59:20Z")

</div>

Thanks

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 25, 2018, 9:59am UTC](https://discuss.elastic.co/t/elasticsearch-and-sql-tables/154261/8 "2018-11-25T09:59:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.