# Updated data from jdbc(mysql) to elasticsearch by using logstash

**URL:** <https://discuss.elastic.co/t/updated-data-from-jdbc-mysql-to-elasticsearch-by-using-logstash/124058>\
**Category:** Logstash\
**Created:** [March 15, 2018, 9:42am UTC](https://discuss.elastic.co/t/updated-data-from-jdbc-mysql-to-elasticsearch-by-using-logstash/124058 "2018-03-15T09:42:09Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![songhee](https://avatars.discourse-cdn.com/v4/letter/s/e36b37/32.png) [@songhee](https://discuss.elastic.co/u/songhee)\
**Post date:** [March 15, 2018, 9:42am UTC](https://discuss.elastic.co/t/updated-data-from-jdbc-mysql-to-elasticsearch-by-using-logstash/124058/1 "2018-03-15T09:42:10Z")

</div>

Hello,  
I am also newbie in elk stack.  
I just connected mysql(DBMS) and elasticsearch by using logstash  
and successed to send bulk data from mysql server.  
I want to manage data on mysql. like create, update, delete process(CUD)  
and I want to use the elasticsearch as a search engine [I need to READ data from mysql on elasticsearch]  
to achieve it, the updated, deleted data should be sent to elasticsearch from mysql perfectly. by using logstash

I searched many times, it is impossible..

really it is?\>??

could you help me out to find best exemple applied these process ?  
or git hub url is also ok

mysql - C(CREATE), U(UPDATE), D(DELETE)  
elastiscearch - R(READ)  
logstash - send updated/deleted data from mysql to elasticsearch

this is my logstash configuration

input {  
jdbc {  
jdbc\_driver\_library =\> "../lib/mysql-connector-java-5.1.33.jar"  
jdbc\_driver\_class =\> "...r"  
jdbc\_connection\_string =\> "..."  
jdbc\_user =\> "..."  
jdbc\_password =\> "..."  
jdbc\_validate\_connection =\> true  
schedule =\> "\* \* \* \* \* \*"   
tracking\_column =\> "id"  
use\_column\_value =\> true  
tracking\_column\_type =\> "numeric"  
clean\_run =\> true  
statement =\> "SELECT id, title, content, lastUpdateDate, ip, userName FROM board where id\> :sql\_last\_value order by id"  
jdbc\_pool\_timeout =\> 10  
jdbc\_paging\_enabled =\> true  
jdbc\_page\_size =\> 10000  
}  
}

output {  
elasticsearch {  
hosts =\> ["localhost:9200"]  
index =\> "board"  
document\_type =\> "posts"  
document\_id =\> "%{id}"  
action =\> "update"  
doc\_as\_upsert =\> true  
}  
stdout {  
codec =\> rubydebug  
}  
}

and i heard that there is a 'upsert' option in output-elasticsearch  
could you explain how to use it?? is it associated with send updated date from jdbc to elasticsearch???

---

<div class="post-metadata">

**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\
**Post date:** [March 17, 2018, 8:27am UTC](https://discuss.elastic.co/t/updated-data-from-jdbc-mysql-to-elasticsearch-by-using-logstash/124058/2 "2018-03-17T08:27:16Z")

</div>

My suspicion is that what you are looking to do isn't quite possible with this architecture.

For example, when a row is deleted from MySQL, the next time the jdbc input runs, it simply won't find that row, and therefore, the stale row will continue to reside in Elasticsearch.

Here is a recent [blog post](https://medium.com/@MentorMate/elasticsearch-how-to-add-full-text-search-to-your-database-ee2f3ea4d3f3) about using Elasticsearch as an app search engine for data in MySQL. It's .NET-focused, so not everything will apply to your use-case exactly, but it should provide the general concepts.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 14, 2018, 8:27am UTC](https://discuss.elastic.co/t/updated-data-from-jdbc-mysql-to-elasticsearch-by-using-logstash/124058/3 "2018-04-14T08:27:25Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.