# Creating, Updating, Validating and Rebuilding Index using Java API **URL:** \ **Category:** Elasticsearch\ **Created:** [January 22, 2013, 8:25am UTC](https://discuss.elastic.co/t/creating-updating-validating-and-rebuilding-index-using-java-api/10448 "2013-01-22T08:25:40Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Mxims\_2](https://avatars.discourse-cdn.com/v4/letter/m/958977/32.png) [@Mxims\_2](https://discuss.elastic.co/u/Mxims_2)\ **Post date:** [January 22, 2013, 8:25am UTC](https://discuss.elastic.co/t/creating-updating-validating-and-rebuilding-index-using-java-api/10448/1 "2013-01-22T08:25:40Z")
Hi - I am using TransportClient to connect to 3 nodes on my cluster. All nodes are identical to each other. Using Java API - - How do I create a new Index with a given type? - Once I create the index, I need to make sure that, it is up to date by keeping with the transactions [CRUD]. I am using the following snippet to add/update the documents into the index. - Using the lines below to delete documents when required - Once in a day, I need to make sure that the index is in sync with my data in the database. To do this verification, I am planning to get all the records from the db, and check whether they are available in the index. Code snippet below.. - SearchRequestBuilder searchBuilder = client.prepareSearch("supplier").setTypes(type); QueryBuilder qb = QueryBuilders.matchAllQuery(); - Now I need to rebuild the index.. with type=Type above. How do I do that? Am I doing it the right way? Would there be any issues with my design? Is deleting the mapping and creating the index with the deleted mapping again causes any trouble? Thank you, Mxims -- ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [January 22, 2013, 8:39am UTC](https://discuss.elastic.co/t/creating-updating-validating-and-rebuilding-index-using-java-api/10448/2 "2013-01-22T08:39:30Z")
I think it's perfectly fine IMHO. That's the way I did it on a previous project. About the way you are going to create index and mappings: If you use Spring, check this: [GitHub - dadoonet/spring-elasticsearch: Spring factories for elasticsearch](https://github.com/dadoonet/spring-elasticsearch) [https://github.com/dadoonet/spring-elasticsearch](https://github.com/dadoonet/spring-elasticsearch) If not, I recommend to create first your index and then to create all mappings you need. Create index : [https://github.com/dadoonet/spring-elasticsearch/blob/master/src/main/java/fr/pilato/spring/elasticsearch/ElasticsearchAbstractClientFactoryBean.java#L703](https://github.com/dadoonet/spring-elasticsearch/blob/master/src/main/java/fr/pilato/spring/elasticsearch/ElasticsearchAbstractClientFactoryBean.java#L703) [https://github.com/dadoonet/spring-elasticsearch/blob/master/src/main/java/fr/pilato/spring/elasticsearch/ElasticsearchAbstractClientFactoryBean.java#L703](https://github.com/dadoonet/spring-elasticsearch/blob/master/src/main/java/fr/pilato/spring/elasticsearch/ElasticsearchAbstractClientFactoryBean.java#L703) Create mapping: [https://github.com/dadoonet/spring-elasticsearch/blob/master/src/main/java/fr/pilato/spring/elasticsearch/ElasticsearchAbstractClientFactoryBean.java#L667](https://github.com/dadoonet/spring-elasticsearch/blob/master/src/main/java/fr/pilato/spring/elasticsearch/ElasticsearchAbstractClientFactoryBean.java#L667) [https://github.com/dadoonet/spring-elasticsearch/blob/master/src/main/java/fr/pilato/spring/elasticsearch/ElasticsearchAbstractClientFactoryBean.java#L667](https://github.com/dadoonet/spring-elasticsearch/blob/master/src/main/java/fr/pilato/spring/elasticsearch/ElasticsearchAbstractClientFactoryBean.java#L667) Does it help? Le 22 janvier 2013 à 09:25, Mxims [renjith.ck@gmail.com](mailto:renjith.ck@gmail.com) a écrit : > Hi - > > I am using TransportClient to connect to 3 nodes on my cluster. All nodes are > identical to each other. > > Using Java API - > \* How do I create a new Index with a given type? > \* Once I create the index, I need to make sure that, it is up to date by > keeping with the transactions [CRUD]. I am using the following snippet to > add/update the documents into the index. > > ``` > bulkRequest.add(client.prepareIndex("supplier", type, id) > .setSource(jsonBuilder() > .startObject() > .field("name", name) > .field("revision", revision) > .field("description", description) > .field("current", current) > .field("kindof", kindof) > .endObject() > ) > ); > > * Using the lines below to delete documents when required > o bulkRequest.add(client.prepareDelete("supplier", type, id)); > * Once in a day, I need to make sure that the index is in sync with my > > ``` > > data in the database. To do this verification, I am planning to get all the > records from the db, and check whether they are available in the index. Code > snippet below.. > o SearchRequestBuilder searchBuilder = > client.prepareSearch("supplier").setTypes(type); > QueryBuilder qb = QueryBuilders.matchAllQuery(); > o SearchResponse scrollResp = > searchBuilder.setSearchType(SearchType.SCAN) > .setScroll(new TimeValue(100)) > .setQuery(qb) > .setSize(50).execute().actionGet(); > while (true) { > scrollResp = > client.prepareSearchScroll(scrollResp.getScrollId()) > .setScroll(new TimeValue(100)) > .execute() > .actionGet(); > > ``` > for (SearchHit hit : scrollResp.getHits()) { > list.add(hit.id()); > > names.add(hit.sourceAsMap().get("name")); > } > > //Break condition: No hits are returned > if (scrollResp.hits().hits().length == 0) { > break; > } > } > * If there is mismatch, I need to rebuild the index from scratch. To do > > ``` > > that I am currently deleting the mapping "type" from the "supplier" index. > o DeleteMappingRequest req = new DeleteMappingRequest("supplier"); > req.type(type); > client.admin().indices().deleteMapping(req).actionGet(); > \* Now I need to rebuild the index.. with type=Type above. How do I do > that? > Am I doing it the right way? Would there be any issues with my design? Is > deleting the mapping and creating the index with the deleted mapping again > causes any trouble? > > Thank you, > Mxims > > -- -- David Pilato [http://www.scrutmydocs.org/](http://www.scrutmydocs.org/) [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs -- ---
**Author:** ![Mxims\_2](https://avatars.discourse-cdn.com/v4/letter/m/958977/32.png) [@Mxims\_2](https://discuss.elastic.co/u/Mxims_2)\ **Post date:** [January 23, 2013, 10:43pm UTC](https://discuss.elastic.co/t/creating-updating-validating-and-rebuilding-index-using-java-api/10448/3 "2013-01-23T22:43:47Z")
Hi David, Thank you for your help. I have been trying it from past two days. Here is my scenario and what is going on with my index.. I have my index created with about 6 fields, all fields are strings. I am using the TYPE as a key to identify different buckets of records. Type can be 1234, 7890, 4567, etc. In my design, TYPE determines the bucket for each records that are indexed. Users are restricted to search only within their assigned bucket(s). i.e., While searching if I specify a given type (for example, type=1234 or type=7890, etc.), result set should only contain records that belong to that type. When I am required to re-index the documents which belong to the type=1234, at first I am dropping that mapping for type=1234 from the index, so that it clears all the records that belong to that type. When I re-index the records, for some reason records are updated in the index without having a type. So I get a type missing exception. I am not sure whether I am supposed to use the TYPE in this fashion to flag different buckets of records. Instead of deleting the type mapping, if I delete records one-by-one from the index, and re-index I don't find any issues. Any advice? Thanks Mxims On Tuesday, January 22, 2013 12:39:30 AM UTC-8, David Pilato wrote: > I think it's perfectly fine IMHO. > That's the way I did it on a previous project. > > About the way you are going to create index and mappings: > If you use Spring, check this: > [GitHub - dadoonet/spring-elasticsearch: Spring factories for elasticsearch](https://github.com/dadoonet/spring-elasticsearch) > If not, I recommend to create first your index and then to create all > mappings you need. > > Create index : > [https://github.com/dadoonet/spring-elasticsearch/blob/master/src/main/java/fr/pilato/spring/elasticsearch/ElasticsearchAbstractClientFactoryBean.java#L703](https://github.com/dadoonet/spring-elasticsearch/blob/master/src/main/java/fr/pilato/spring/elasticsearch/ElasticsearchAbstractClientFactoryBean.java#L703) > Create mapping: > [https://github.com/dadoonet/spring-elasticsearch/blob/master/src/main/java/fr/pilato/spring/elasticsearch/ElasticsearchAbstractClientFactoryBean.java#L667](https://github.com/dadoonet/spring-elasticsearch/blob/master/src/main/java/fr/pilato/spring/elasticsearch/ElasticsearchAbstractClientFactoryBean.java#L667) > > Does it help? > > Le 22 janvier 2013 à 09:25, Mxims \<[renji...@gmail.com](mailto:renji...@gmail.com) \\> a > écrit : > > Hi - > > I am using TransportClient to connect to 3 nodes on my cluster. All nodes > are identical to each other. > > Using Java API - > > - How do I create a new Index with a given type? > > - Once I create the index, I need to make sure that, it is up to date > by keeping with the transactions [CRUD]. I am using the following snippet > to add/update the documents into the index. > > - Using the lines below to delete documents when required > > - Once in a day, I need to make sure that the index is in sync with my > data in the database. To do this verification, I am planning to get all the > records from the db, and check whether they are available in the index. > Code snippet below.. > > - If there is mismatch, I need to rebuild the index from scratch. To > do that I am currently deleting the mapping "type" from the "supplier" > index. > > - Now I need to rebuild the index.. with type=Type above. How do I do > that? > > Am I doing it the right way? Would there be any issues with my design? Is > deleting the mapping and creating the index with the deleted mapping again > causes any trouble? > > Thank you, > Mxims > > -- > > -- > David Pilato > [http://www.scrutmydocs.org/](http://www.scrutmydocs.org/) > [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/) > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs -- ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [January 24, 2013, 4:39pm UTC](https://discuss.elastic.co/t/creating-updating-validating-and-rebuilding-index-using-java-api/10448/4 "2013-01-24T16:39:34Z")
Hiya Mxims > I have my index created with about 6 fields, all fields are strings. I > am using the TYPE as a key to identify different buckets of records. > Type can be 1234, 7890, 4567, etc. In my design, TYPE determines the > bucket for each records that are indexed. Users are restricted to > search only within their assigned bucket(s). i.e., While searching if > I specify a given type (for example, type=1234 or type=7890, etc.), > result set should only contain records that belong to that type. > > When I am required to re-index the documents which belong to the > type=1234, at first I am dropping that mapping for type=1234 from the > index, so that it clears all the records that belong to that type. > When I re-index the records, for some reason records are updated in > the index without having a type. So I get a type missing exception. I > am not sure whether I am supposed to use the TYPE in this fashion to > flag different buckets of records. > > Instead of deleting the type mapping, if I delete records one-by-one > from the index, and re-index I don't find any issues. I tried the process you described above, and it all seems to work normally. Possibly we're doing different things. Please can you gist a curl recreation of what you're actually doing, to show the problem that you encounter. thanks Clint -- ---
**Author:** ![Mxims\_2](https://avatars.discourse-cdn.com/v4/letter/m/958977/32.png) [@Mxims\_2](https://discuss.elastic.co/u/Mxims_2)\ **Post date:** [January 29, 2013, 1:31am UTC](https://discuss.elastic.co/t/creating-updating-validating-and-rebuilding-index-using-java-api/10448/5 "2013-01-29T01:31:54Z")
Hi Clint, It seems the issue was related to deleting the mapping.. after I deleted the mapping, I was not creating the mapping again before I indexed the new documents. After I did the put mapping, things seem to be working fine! Thanks a lot for your help! Thanks On Thursday, January 24, 2013 8:39:34 AM UTC-8, Clinton Gormley wrote: > Hiya Mxims > > > I have my index created with about 6 fields, all fields are strings. I > > am using the TYPE as a key to identify different buckets of records. > > Type can be 1234, 7890, 4567, etc. In my design, TYPE determines the > > bucket for each records that are indexed. Users are restricted to > > search only within their assigned bucket(s). i.e., While searching if > > I specify a given type (for example, type=1234 or type=7890, etc.), > > result set should only contain records that belong to that type. > > > > When I am required to re-index the documents which belong to the > > type=1234, at first I am dropping that mapping for type=1234 from the > > index, so that it clears all the records that belong to that type. > > When I re-index the records, for some reason records are updated in > > the index without having a type. So I get a type missing exception. I > > am not sure whether I am supposed to use the TYPE in this fashion to > > flag different buckets of records. > > > > Instead of deleting the type mapping, if I delete records one-by-one > > from the index, and re-index I don't find any issues. > > I tried the process you described above, and it all seems to work > normally. Possibly we're doing different things. > > Please can you gist a curl recreation of what you're actually doing, to > show the problem that you encounter. > > thanks > > Clint -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:54am UTC](https://discuss.elastic.co/t/creating-updating-validating-and-rebuilding-index-using-java-api/10448/6 "2017-07-06T02:54:17Z")