# Elasticsearch- Single Index vs Multiple Indexes

**URL:** <https://discuss.elastic.co/t/elasticsearch-single-index-vs-multiple-indexes/164265>\
**Category:** Elasticsearch\
**Created:** [January 15, 2019, 8:39am UTC](https://discuss.elastic.co/t/elasticsearch-single-index-vs-multiple-indexes/164265 "2019-01-15T08:39:33Z")\
**Posts on this page:** 6\
**Page:** 3

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [February 4, 2019, 8:04am UTC](https://discuss.elastic.co/t/elasticsearch-single-index-vs-multiple-indexes/164265/42 "2019-02-04T08:04:59Z")

</div>

Try dropping clauses to find the costs.

> [@pranav24](#):
>
> "slop":"10000",

Can you really consider something a phrase if the words are ten thousand words apart?

---

<div class="post-metadata">

**Author:** ![pranav24](https://avatars.discourse-cdn.com/v4/letter/p/b19c9b/32.png) [@pranav24](https://discuss.elastic.co/u/pranav24)\
**Post date:** [February 5, 2019, 9:14am UTC](https://discuss.elastic.co/t/elasticsearch-single-index-vs-multiple-indexes/164265/43 "2019-02-05T09:14:19Z")

</div>

Thanks Mark for your response.  
The slop of 10000 was for a different purpose. However, I tested and found that it is not the issue.

As per your suggestion, I delved deeper and found that search on large number of fields was leading to too many disk seeks which is actually the real issue.

I am planning to go ahead with a single copy\_to field to reduce the number of fields being searched upon. However, I have 2 further questions:

1. Using copy\_to how can we be able to highlight the matched text for search?

2. Using copy\_to how can we be able to identify the actual field for which the search text matched.

For example: I have two fields "First\_name" and "Last\_name" both of which are copied to "Full\_name". I search a string present in "First\_name" using "Full\_name". What is the way to find that search string match was part of "First\_name" field?

---

<div class="post-metadata">

**Author:** ![bugggbear](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bugggbear/32/37615_2.png) [@bugggbear](https://discuss.elastic.co/u/bugggbear)\
**Post date:** [February 5, 2019, 4:32pm UTC](https://discuss.elastic.co/t/elasticsearch-single-index-vs-multiple-indexes/164265/44 "2019-02-05T16:32:42Z")

</div>

I'm not sure how relevant would be this suggestion, but I'm going to put it here 🙂

About the mapping explosion, I had similiar issue, after trying to store the names of different Cookie fields in a separate index field, and their values as fied values. I had no idea how much 'random' cookie names would come to my index, which would definitely lead to mapping explosion.

I ended up with the following solution:

My wrong index mapping, was looking like this :  
index : {  
"$cookie\_name" : "$cookie\_value"  
"$cookie\_name2" : "$cookie\_value2"  
}

The right solution was to use nested data type and the following structure :  
Index: {  
Cookie:  
[  
{  
cookie\_name : $cookie\_name1,  
cookie\_value : $cookie\_value1  
},  
{  
cookie\_name : $cookie\_name2,  
cookie\_value : $cookie\_value2  
},  
]  
}

With this structure, I'm now able to have infinite number of different cookie names , without risking to hit mapping explosion.

Here is some info on the nested datatype:  
[https://www.elastic.co/guide/en/elasticsearch/reference/current/nested.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/nested.html)

---

<div class="post-metadata">

**Author:** ![pranav24](https://avatars.discourse-cdn.com/v4/letter/p/b19c9b/32.png) [@pranav24](https://discuss.elastic.co/u/pranav24)\
**Post date:** [February 6, 2019, 7:52am UTC](https://discuss.elastic.co/t/elasticsearch-single-index-vs-multiple-indexes/164265/45 "2019-02-06T07:52:25Z")

</div>

Thank you @bugggbear for the response.

> [@bugggbear](#):
>
> The right solution was to use nested data type and the following structure :  
> Index: {  
> Cookie:  
> [  
> {  
> cookie\_name : $cookie\_name1,  
> cookie\_value : $cookie\_value1  
> },  
> {  
> cookie\_name : $cookie\_name2,  
> cookie\_value : $cookie\_value2  
> },  
> ]  
> }

Although this seems to reduce mapping explosion issue but have the following questions:

1. This approach would fail to retrieve field information at the time of highlighting i.e. when I search on **cookie\_value1** , how do I ensure it belongs to **cookie\_name1**. I need this information as **cookie\_value1** is actually a value of the key **cookie\_name1**.

2. How do I achieve an update of a particular part of this Cookie array? Lets say, I will have to update **cookie\_value** from **cookie\_value2** to **cookie\_value3**.

3. Sometimes cookie\_value might contain an array of strings. For example, In my use case **cookie\_value** may be of different types. It can be an array of strings too. Lets say it contains [" **cookie\_value4**", " **cookie\_value5**"] . How will this scenario be taken care internally by Elasicsearch?

4. How can we be able to achieve features like aggregation, sorting and filtering?

---

<div class="post-metadata">

**Author:** ![bugggbear](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bugggbear/32/37615_2.png) [@bugggbear](https://discuss.elastic.co/u/bugggbear)\
**Post date:** [February 6, 2019, 8:15am UTC](https://discuss.elastic.co/t/elasticsearch-single-index-vs-multiple-indexes/164265/46 "2019-02-06T08:15:57Z")

</div>

> [@pranav24](#):
>
> This approach would fail to retrieve field information at the time of highlighting i.e. when I search on **cookie\_value1** , how do I ensure it belongs to **cookie\_name1**. I need this information as **cookie\_value1** is actually a value of the key **cookie\_name1**.

I'm not quite sure about 1)

For the rest of your answers, I think the following links have most of the answers:  
[https://qbox.io/blog/sorting-nested-fields-in-elasticsearch](https://qbox.io/blog/sorting-nested-fields-in-elasticsearch)  
[https://qbox.io/blog/elasticsearch-aggregations-nested-documents-tutorial](https://qbox.io/blog/elasticsearch-aggregations-nested-documents-tutorial)

> **[Nested field type | Elasticsearch Guide \[8.11\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/nested.html)**

I have only used basic filtering by cookie name till now. I suppose someone more experienced with nested documents could give you more deep answers regarding your questions.

Cheers.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 6, 2019, 8:16am UTC](https://discuss.elastic.co/t/elasticsearch-single-index-vs-multiple-indexes/164265/47 "2019-03-06T08:16:03Z")

</div>

Closed on March 6, 2019, 8:16am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

[Previous page](https://discuss.elastic.co/t/elasticsearch-single-index-vs-multiple-indexes/164265.md?page=2)