# Implementing ES index creation and search in parallel on the same node **URL:** \ **Category:** Elasticsearch\ **Created:** [April 28, 2017, 5:54am UTC](https://discuss.elastic.co/t/implementing-es-index-creation-and-search-in-parallel-on-the-same-node/83967 "2017-04-28T05:54:22Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Bhargavi\_Sri](https://avatars.discourse-cdn.com/v4/letter/b/b38774/32.png) [@Bhargavi\_Sri](https://discuss.elastic.co/u/Bhargavi_Sri)\ **Post date:** [April 28, 2017, 5:54am UTC](https://discuss.elastic.co/t/implementing-es-index-creation-and-search-in-parallel-on-the-same-node/83967/1 "2017-04-28T05:54:23Z")
Hi guys. I really am trying to understand Elasticsearch. The code that I have written creates 35 parallel python scripts that all try to create indices on ES cluster simultaneously. But this is happening in serial and I am unable to implement parallelism to make my code run faster. Is this because ES does not allow multiple scripts to index at once? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [April 28, 2017, 6:34am UTC](https://discuss.elastic.co/t/implementing-es-index-creation-and-search-in-parallel-on-the-same-node/83967/2 "2017-04-28T06:34:48Z")
Indexing can be done in parallel by multiple threads, but creating indices have to update the cluster state and therefore need to be serialised through the master node. ---
**Author:** ![Bhargavi\_Sri](https://avatars.discourse-cdn.com/v4/letter/b/b38774/32.png) [@Bhargavi\_Sri](https://discuss.elastic.co/u/Bhargavi_Sri)\ **Post date:** [April 28, 2017, 7:12am UTC](https://discuss.elastic.co/t/implementing-es-index-creation-and-search-in-parallel-on-the-same-node/83967/3 "2017-04-28T07:12:53Z")
But in my code I have the creation of index and searching linked together. So is there nothing I can do in this case? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [April 28, 2017, 7:36am UTC](https://discuss.elastic.co/t/implementing-es-index-creation-and-search-in-parallel-on-the-same-node/83967/4 "2017-04-28T07:36:20Z")
Creating indices is usually not done very frequently, so the fact that the cluster state need to be updated is generally not a problem. What exactly is it you are trying to achieve? ---
**Author:** ![Bhargavi\_Sri](https://avatars.discourse-cdn.com/v4/letter/b/b38774/32.png) [@Bhargavi\_Sri](https://discuss.elastic.co/u/Bhargavi_Sri)\ **Post date:** [April 28, 2017, 7:44am UTC](https://discuss.elastic.co/t/implementing-es-index-creation-and-search-in-parallel-on-the-same-node/83967/5 "2017-04-28T07:44:52Z")
I do not have the need to store the data on the node. So I am creating indices of size 10k, performing the search on them and then deleting it before I create another index. Since match query is not able to give me all the results of the search query, I am creating the index on 10k records and retrieving all the results before I move on to create another index. For example, if the data size is 20k, I am creating 2 indices through a for-loop. Deleting the first one after the search is done ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 26, 2017, 7:58am UTC](https://discuss.elastic.co/t/implementing-es-index-creation-and-search-in-parallel-on-the-same-node/83967/6 "2017-05-26T07:58:56Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.