# Updating the indexes in every half an hour
**URL:** https://discuss.elastic.co/t/updating-the-indexes-in-every-half-an-hour/15688
**Category:** Elasticsearch
**Created:** [February 10, 2014, 9:40am UTC](https://discuss.elastic.co/t/updating-the-indexes-in-every-half-an-hour/15688 "2014-02-10T09:40:31Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@vallabh](https://discuss.elastic.co/u/vallabh)
#### Post date: [February 10, 2014, 9:40am UTC](https://discuss.elastic.co/t/updating-the-indexes-in-every-half-an-hour/15688/1 "2014-02-10T09:40:31Z")
Dear All,
I am using elasticsearch in some of the my API.
I have created the index and document and have added data in elasticsearch
server from Mysql database.
I am following 3 steps that is,
1. Delete the index using
curl -X DELETE '[http://localhost:9200/adminvenue/?pretty=true](http://localhost:9200/adminvenue/?pretty=true)'
2. Create the index using
curl -X PUT '[http://localhost:9200/adminvenue/?pretty=true](http://localhost:9200/adminvenue/?pretty=true)' -d '
{.......
3. Create the mapping using
curl -X PUT
'[http://localhost:9200/adminvenue/jos\_content/\_mapping?pretty=true](http://localhost:9200/adminvenue/jos_content/_mapping?pretty=true)' -d '
{......
I have set the cron which runs every half an hour.
Cron consist of 2 files named .sh file which is for
creating/deleting/mapping indexes and .php which is use for adding data
from mysql to elasticsearch.
My mysql data is continuously updating that is some data get removed some
modified and some added because of this i have run cron for 1/2 hour.
My concerns is that i have to first delete the indexes and then create the
indexes in every 1/2 hour.
Is there any way to update the indexes that is i just wanted to update the
data in elasticsearch which is added, deleted and modified in mysql ?
Any suggestion is very much appreciated.
Thanks,
Vallabh
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/718b0eb6-048e-46bf-95f4-683389d24496%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/718b0eb6-048e-46bf-95f4-683389d24496%40googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Kevin\_Wang](https://discuss.elastic.co/u/Kevin_Wang)
#### Post date: [February 10, 2014, 10:08am UTC](https://discuss.elastic.co/t/updating-the-indexes-in-every-half-an-hour/15688/2 "2014-02-10T10:08:28Z")
You can use JDBC river plugin to fetch from database directly.
> **[GitHub - jprante/elasticsearch-jdbc: JDBC importer for Elasticsearch](https://github.com/jprante/elasticsearch-jdbc)**
>
> JDBC importer for Elasticsearch. Contribute to jprante/elasticsearch-jdbc development by creating an account on GitHub.
On Monday, February 10, 2014 8:40:31 PM UTC+11, Vallabh Bothre wrote:
> Dear All,
>
> I am using elasticsearch in some of the my API.
> I have created the index and document and have added data in elasticsearch
> server from Mysql database.
>
> I am following 3 steps that is,
>
> 1. Delete the index using
> curl -X DELETE '[http://localhost:9200/adminvenue/?pretty=true](http://localhost:9200/adminvenue/?pretty=true)'
>
> 2. Create the index using
> curl -X PUT '[http://localhost:9200/adminvenue/?pretty=true](http://localhost:9200/adminvenue/?pretty=true)' -d '
> {.......
>
> 3. Create the mapping using
> curl -X PUT '
> [http://localhost:9200/adminvenue/jos\_content/\_mapping?pretty=true](http://localhost:9200/adminvenue/jos_content/_mapping?pretty=true)' -d '
> {......
>
> I have set the cron which runs every half an hour.
> Cron consist of 2 files named .sh file which is for
> creating/deleting/mapping indexes and .php which is use for adding data
> from mysql to elasticsearch.
>
> My mysql data is continuously updating that is some data get removed some
> modified and some added because of this i have run cron for 1/2 hour.
>
> My concerns is that i have to first delete the indexes and then create the
> indexes in every 1/2 hour.
> Is there any way to update the indexes that is i just wanted to update the
> data in elasticsearch which is added, deleted and modified in mysql ?
>
> Any suggestion is very much appreciated.
>
> Thanks,
> Vallabh
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/05415e13-f76c-420e-bac8-15a754230296%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/05415e13-f76c-420e-bac8-15a754230296%40googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@vallabh](https://discuss.elastic.co/u/vallabh)
#### Post date: [February 10, 2014, 1:41pm UTC](https://discuss.elastic.co/t/updating-the-indexes-in-every-half-an-hour/15688/3 "2014-02-10T13:41:39Z")
Thank you Kevin for your reply,
But i was searching for the command which will directly update the data
without deleting and creating the indexes.
Is there any command like curl -X UPDATE in elasticsearch ?
which will not add already existing data in elastic.
On Monday, February 10, 2014 3:38:28 PM UTC+5:30, Kevin Wang wrote:
> You can use JDBC river plugin to fetch from database directly.
>
> [GitHub - jprante/elasticsearch-jdbc: JDBC importer for Elasticsearch](https://github.com/jprante/elasticsearch-river-jdbc)
>
> On Monday, February 10, 2014 8:40:31 PM UTC+11, Vallabh Bothre wrote:
>
> > Dear All,
> >
> > I am using elasticsearch in some of the my API.
> > I have created the index and document and have added data in
> > elasticsearch server from Mysql database.
> >
> > I am following 3 steps that is,
> >
> > 1. Delete the index using
> > curl -X DELETE '[http://localhost:9200/adminvenue/?pretty=true](http://localhost:9200/adminvenue/?pretty=true)'
> >
> > 2. Create the index using
> > curl -X PUT '[http://localhost:9200/adminvenue/?pretty=true](http://localhost:9200/adminvenue/?pretty=true)' -d '
> > {.......
> >
> > 3. Create the mapping using
> > curl -X PUT '
> > [http://localhost:9200/adminvenue/jos\_content/\_mapping?pretty=true](http://localhost:9200/adminvenue/jos_content/_mapping?pretty=true)' -d '
> > {......
> >
> > I have set the cron which runs every half an hour.
> > Cron consist of 2 files named .sh file which is for
> > creating/deleting/mapping indexes and .php which is use for adding data
> > from mysql to elasticsearch.
> >
> > My mysql data is continuously updating that is some data get removed some
> > modified and some added because of this i have run cron for 1/2 hour.
> >
> > My concerns is that i have to first delete the indexes and then create
> > the indexes in every 1/2 hour.
> > Is there any way to update the indexes that is i just wanted to update
> > the data in elasticsearch which is added, deleted and modified in mysql ?
> >
> > Any suggestion is very much appreciated.
> >
> > Thanks,
> > Vallabh
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/1f8c781f-2a67-48ee-8f77-bd38a3d8a40e%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1f8c781f-2a67-48ee-8f77-bd38a3d8a40e%40googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Binh\_Ly](https://discuss.elastic.co/u/Binh_Ly)
#### Post date: [February 10, 2014, 1:48pm UTC](https://discuss.elastic.co/t/updating-the-indexes-in-every-half-an-hour/15688/4 "2014-02-10T13:48:06Z")
Vallabh,
You can do full document updates/replace by simply using POST
[http://server:9200/](http://server:9200/)//id. Just specify the document ID of the
document you want to update and it will replace that document.
You can also do partial updates of an existing document like this:
[http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-update.html](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-update.html)
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/db5dd077-5af5-4b0b-aef6-6f080141a6f7%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/db5dd077-5af5-4b0b-aef6-6f080141a6f7%40googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Randall\_McRee](https://discuss.elastic.co/u/Randall_McRee)
#### Post date: [February 10, 2014, 7:06pm UTC](https://discuss.elastic.co/t/updating-the-indexes-in-every-half-an-hour/15688/5 "2014-02-10T19:06:59Z")
You can use an alias to enable switching between two indexes. Therefore,
you can build a new index while allowing searching against the old one.
When the new one is successfully built change the alias to point to the new
one.
On Mon, Feb 10, 2014 at 1:40 AM, Vallabh Bothre [vallabh.bothre@gmail.com](mailto:vallabh.bothre@gmail.com)wrote:
> Dear All,
>
> I am using elasticsearch in some of the my API.
> I have created the index and document and have added data in elasticsearch
> server from Mysql database.
>
> I am following 3 steps that is,
>
> 1. Delete the index using
> curl -X DELETE '[http://localhost:9200/adminvenue/?pretty=true](http://localhost:9200/adminvenue/?pretty=true)'
>
> 2. Create the index using
> curl -X PUT '[http://localhost:9200/adminvenue/?pretty=true](http://localhost:9200/adminvenue/?pretty=true)' -d '
> {.......
>
> 3. Create the mapping using
> curl -X PUT '
> [http://localhost:9200/adminvenue/jos\_content/\_mapping?pretty=true](http://localhost:9200/adminvenue/jos_content/_mapping?pretty=true)' -d '
> {......
>
> I have set the cron which runs every half an hour.
> Cron consist of 2 files named .sh file which is for
> creating/deleting/mapping indexes and .php which is use for adding data
> from mysql to elasticsearch.
>
> My mysql data is continuously updating that is some data get removed some
> modified and some added because of this i have run cron for 1/2 hour.
>
> My concerns is that i have to first delete the indexes and then create the
> indexes in every 1/2 hour.
> Is there any way to update the indexes that is i just wanted to update the
> data in elasticsearch which is added, deleted and modified in mysql ?
>
> Any suggestion is very much appreciated.
>
> Thanks,
> Vallabh
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> To view this discussion on the web visit
> [https://groups.google.com/d/msgid/elasticsearch/718b0eb6-048e-46bf-95f4-683389d24496%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/718b0eb6-048e-46bf-95f4-683389d24496%40googlegroups.com)
> .
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAFjHw35KEsrd6J\_pC3SjtWx5KbV5xKPBXpvPNzsNGff6d45S7w%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAFjHw35KEsrd6J_pC3SjtWx5KbV5xKPBXpvPNzsNGff6d45S7w%40mail.gmail.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@vallabh](https://discuss.elastic.co/u/vallabh)
#### Post date: [February 11, 2014, 5:52am UTC](https://discuss.elastic.co/t/updating-the-indexes-in-every-half-an-hour/15688/6 "2014-02-11T05:52:45Z")
Thanks RKM.
On Tuesday, February 11, 2014 12:36:59 AM UTC+5:30, RKM wrote:
> You can use an alias to enable switching between two indexes. Therefore,
> you can build a new index while allowing searching against the old one.
> When the new one is successfully built change the alias to point to the new
> one.
>
> On Mon, Feb 10, 2014 at 1:40 AM, Vallabh Bothre \<[vallabh...@gmail.com](mailto:vallabh...@gmail.com)\
>
> > wrote:
>
> > Dear All,
> >
> > I am using elasticsearch in some of the my API.
> > I have created the index and document and have added data in
> > elasticsearch server from Mysql database.
> >
> > I am following 3 steps that is,
> >
> > 1. Delete the index using
> > curl -X DELETE '[http://localhost:9200/adminvenue/?pretty=true](http://localhost:9200/adminvenue/?pretty=true)'
> >
> > 2. Create the index using
> > curl -X PUT '[http://localhost:9200/adminvenue/?pretty=true](http://localhost:9200/adminvenue/?pretty=true)' -d '
> > {.......
> >
> > 3. Create the mapping using
> > curl -X PUT '
> > [http://localhost:9200/adminvenue/jos\_content/\_mapping?pretty=true](http://localhost:9200/adminvenue/jos_content/_mapping?pretty=true)' -d '
> > {......
> >
> > I have set the cron which runs every half an hour.
> > Cron consist of 2 files named .sh file which is for
> > creating/deleting/mapping indexes and .php which is use for adding data
> > from mysql to elasticsearch.
> >
> > My mysql data is continuously updating that is some data get removed some
> > modified and some added because of this i have run cron for 1/2 hour.
> >
> > My concerns is that i have to first delete the indexes and then create
> > the indexes in every 1/2 hour.
> > Is there any way to update the indexes that is i just wanted to update
> > the data in elasticsearch which is added, deleted and modified in mysql ?
> >
> > Any suggestion is very much appreciated.
> >
> > Thanks,
> > Vallabh
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "elasticsearch" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \.
> > To view this discussion on the web visit
> > [https://groups.google.com/d/msgid/elasticsearch/718b0eb6-048e-46bf-95f4-683389d24496%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/718b0eb6-048e-46bf-95f4-683389d24496%40googlegroups.com)
> > .
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/82a37167-7061-42f6-88f3-8bf89ce07c5c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/82a37167-7061-42f6-88f3-8bf89ce07c5c%40googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@jprante](https://discuss.elastic.co/u/jprante)
#### Post date: [February 11, 2014, 8:03am UTC](https://discuss.elastic.co/t/updating-the-indexes-in-every-half-an-hour/15688/7 "2014-02-11T08:03:52Z")
For streams (changing data over time with timestamps), the simplest method
in ES is really to manage timeframe-based indexes, like in the logstash
model. So, you can decide to drop indexes when they become too old. There
are tools that can help in this job, the curator.
By using index aliases, there is no disadvantage in comparison to a single
index. And, ES index drops are very fast, and they do not require resources
like document deletes which allocate space in segments. This should be
carefully considered.
With JDBC river, there is no requirement to delete and create the indexes,
just use the same \_id to address your docs, by a crontab definition. It is
very easy to run this every 30 minutes. The docs will get overwritten and
receive a new version number.
Aside form the river, by checking the version of the docs in ES, you can
find out old documents. The JDBC river does no longer take the job of
deleting old documents, this has to be done from outside.
Jörg
On Mon, Feb 10, 2014 at 2:41 PM, Vallabh Bothre [vallabh.bothre@gmail.com](mailto:vallabh.bothre@gmail.com)wrote:
> Thank you Kevin for your reply,
>
> But i was searching for the command which will directly update the data
> without deleting and creating the indexes.
>
> Is there any command like curl -X UPDATE in elasticsearch ?
> which will not add already existing data in elastic.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHGMWY78d\_qgt-P8VQCV9inDMKZUgpBir3\_APh%3DHERpHw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHGMWY78d_qgt-P8VQCV9inDMKZUgpBir3_APh%3DHERpHw%40mail.gmail.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:51am UTC](https://discuss.elastic.co/t/updating-the-indexes-in-every-half-an-hour/15688/8 "2017-07-06T01:51:12Z")