# How to I migrate my existing postgres data to elasticsearch and also keep it in sync? **URL:** \ **Category:** Elasticsearch\ **Created:** [October 28, 2018, 6:16am UTC](https://discuss.elastic.co/t/how-to-i-migrate-my-existing-postgres-data-to-elasticsearch-and-also-keep-it-in-sync/154314 "2018-10-28T06:16:04Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![patrick007](https://avatars.discourse-cdn.com/v4/letter/p/4af34b/32.png) [@patrick007](https://discuss.elastic.co/u/patrick007)\ **Post date:** [October 28, 2018, 6:16am UTC](https://discuss.elastic.co/t/how-to-i-migrate-my-existing-postgres-data-to-elasticsearch-and-also-keep-it-in-sync/154314/1 "2018-10-28T06:16:04Z")
I want to migrate my existing postgres db data to elasticsearch. And after the migration also want to update the elasticsearch with new update to the database ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [October 28, 2018, 9:21am UTC](https://discuss.elastic.co/t/how-to-i-migrate-my-existing-postgres-data-to-elasticsearch-and-also-keep-it-in-sync/154314/2 "2018-10-28T09:21:34Z")
I shared most of my thoughts there: [http://david.pilato.fr/blog/2015/05/09/advanced-search-for-your-legacy-application/](http://david.pilato.fr/blog/2015/05/09/advanced-search-for-your-legacy-application/) Basically, I'd recommend modifying the application layer if possible and send data to elasticsearch in the same "transaction" as you are sending your data to the database. ---
**Author:** ![patrick007](https://avatars.discourse-cdn.com/v4/letter/p/4af34b/32.png) [@patrick007](https://discuss.elastic.co/u/patrick007)\ **Post date:** [October 28, 2018, 10:26am UTC](https://discuss.elastic.co/t/how-to-i-migrate-my-existing-postgres-data-to-elasticsearch-and-also-keep-it-in-sync/154314/3 "2018-10-28T10:26:22Z")
Can i use logstash ? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [October 28, 2018, 10:44am UTC](https://discuss.elastic.co/t/how-to-i-migrate-my-existing-postgres-data-to-elasticsearch-and-also-keep-it-in-sync/154314/4 "2018-10-28T10:44:56Z")
Yes with the jdbc input plugin. ---
**Author:** ![patrick007](https://avatars.discourse-cdn.com/v4/letter/p/4af34b/32.png) [@patrick007](https://discuss.elastic.co/u/patrick007)\ **Post date:** [October 28, 2018, 2:09pm UTC](https://discuss.elastic.co/t/how-to-i-migrate-my-existing-postgres-data-to-elasticsearch-and-also-keep-it-in-sync/154314/5 "2018-10-28T14:09:35Z")
Okay Thanks, Can I have realtime sync using Logstash ? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [October 28, 2018, 3:05pm UTC](https://discuss.elastic.co/t/how-to-i-migrate-my-existing-postgres-data-to-elasticsearch-and-also-keep-it-in-sync/154314/6 "2018-10-28T15:05:45Z")
No you can't with the jdbc input. ---
**Author:** ![Sjaak01](https://avatars.discourse-cdn.com/v4/letter/s/73ab20/32.png) [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)\ **Post date:** [October 29, 2018, 6:03am UTC](https://discuss.elastic.co/t/how-to-i-migrate-my-existing-postgres-data-to-elasticsearch-and-also-keep-it-in-sync/154314/7 "2018-10-29T06:03:38Z")
Can it do incremental updates? E.g. you rung the jdbc every hour and then it pulls all the data from postgres from the last hour into Elastic? I'm looking at the same problem. In my case old documents won't change though, just need to pull in the new data at x interval. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [October 29, 2018, 6:50am UTC](https://discuss.elastic.co/t/how-to-i-migrate-my-existing-postgres-data-to-elasticsearch-and-also-keep-it-in-sync/154314/8 "2018-10-29T06:50:54Z")
> [@Sjaak01](#): > > Can it do incremental updates? Yes. As long as you have something like a `lastUpdated` field in your database, you can probably select all the records `WHERE lastUpdated > ` "now". Then if you are generating elasticsearch document's `_id` from one of the fields of your database entity, Elasticsearch will overwrite the existing version by the new one. ---
**Author:** ![patrick007](https://avatars.discourse-cdn.com/v4/letter/p/4af34b/32.png) [@patrick007](https://discuss.elastic.co/u/patrick007)\ **Post date:** [October 29, 2018, 7:50am UTC](https://discuss.elastic.co/t/how-to-i-migrate-my-existing-postgres-data-to-elasticsearch-and-also-keep-it-in-sync/154314/9 "2018-10-29T07:50:58Z")
Can you suggest something that will help me in keeping the ES sync with my Relational DB ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [October 29, 2018, 7:59am UTC](https://discuss.elastic.co/t/how-to-i-migrate-my-existing-postgres-data-to-elasticsearch-and-also-keep-it-in-sync/154314/10 "2018-10-29T07:59:10Z")
You should read the link David posted. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 26, 2018, 8:14am UTC](https://discuss.elastic.co/t/how-to-i-migrate-my-existing-postgres-data-to-elasticsearch-and-also-keep-it-in-sync/154314/11 "2018-11-26T08:14:12Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.