# Mechanism of internal search with multiple indices

**URL:** <https://discuss.elastic.co/t/mechanism-of-internal-search-with-multiple-indices/16399>\
**Category:** Elasticsearch\
**Created:** [March 17, 2014, 7:30am UTC](https://discuss.elastic.co/t/mechanism-of-internal-search-with-multiple-indices/16399 "2014-03-17T07:30:16Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![golchhamohit](https://avatars.discourse-cdn.com/v4/letter/g/e19adc/32.png) [@golchhamohit](https://discuss.elastic.co/u/golchhamohit)\
**Post date:** [March 17, 2014, 7:30am UTC](https://discuss.elastic.co/t/mechanism-of-internal-search-with-multiple-indices/16399/1 "2014-03-17T07:30:16Z")

</div>

Hi ,

```
  When a query is supplied with multiple indices (having same

```

structure), multiple types, along with some data to be searched, how does  
it work internally ? Does it do a m\*n comparisons where m is the data list  
to be searched and n is the no of indices , or is there any other mechanism  
?  
Can you please explain the mechanism involved in it. ? I reckon there is  
some other way working internally. Thanks.

--

Thanks & Regards,  
Mohit Golchha

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAGXH0MgqvpPwC2LC-%2BzMr5K%3DpBDX39NSt4P3bQmz0U1Q2wXUmg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGXH0MgqvpPwC2LC-%2BzMr5K%3DpBDX39NSt4P3bQmz0U1Q2wXUmg%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\
**Post date:** [March 17, 2014, 12:26pm UTC](https://discuss.elastic.co/t/mechanism-of-internal-search-with-multiple-indices/16399/2 "2014-03-17T12:26:25Z")

</div>

Hi Mohit

All documents stored in a single index are stored at the same "level",  
regardless of their type. The "\_type" is just a hidden field in each  
document. So if you do a search like:

```
GET /index_one,index_two/_search
{ "query": { "match": { "field_foo": "some search terms" }}}

```

then it queries each shard in index\_one and index\_two and reduces the  
results. For each index, it looks for "field\_foo" in all of the types  
defined in that index and uses the mapping for the first "field\_foo" that  
it finds. (This is why fields with the same name but in different types  
should be mapped in the same way).

If you do a search like this:

```
GET /index_on,index_two/type_one,type_two/_search
{ "query": { "match": { "field_foo": "some search terms" }}}

```

then everything works in the same way except:

1. it looks only in type\_one and type\_two for "field\_foo" (and uses the  
first one that it finds)
2. it adds a filter like { "terms": { "\_type": ["type\_one", "type\_two"]}}

If you do a search like this:

```
GET /index_on,index_two/type_one,type_two/_search
{ "query": {
      "multi_match": {
          "fields: ["type_one.field_foo", "type_two.field_foo"]
          "query": "some search terms"
      }
}

```

the the mapping for type\_one.field\_foo and type\_two.field\_foo are  
considered independently, and the query is rewritten to look something like  
this:

{  
"query": {  
"dis\_max": {  
"queries": [  
{  
"filtered": {  
"query": { "match": {"field\_foo": "some search terms" }},  
"filter": { "term": { "\_type": "type\_one" }}  
}  
},  
{  
"filtered": {  
"query": { "match": {"field\_foo": "some search terms" }},  
"filter": { "term": { "\_type": "type\_one" }}  
}  
}  
]  
}  
}  
}

clint

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAPt3XKS37RArKVPM%3DnZjEs4roDHV89W2GUUxNjnqaAxD9ri2jw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPt3XKS37RArKVPM%3DnZjEs4roDHV89W2GUUxNjnqaAxD9ri2jw%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\
**Post date:** [March 17, 2014, 12:26pm UTC](https://discuss.elastic.co/t/mechanism-of-internal-search-with-multiple-indices/16399/3 "2014-03-17T12:26:47Z")

</div>

On 17 March 2014 13:26, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com) wrote:

> "query": {  
> "dis\_max": {  
> "queries": [  
> {  
> "filtered": {  
> "query": { "match": {"field\_foo": "some search terms" }},  
> "filter": { "term": { "\_type": "type\_one" }}  
> }  
> },  
> {  
> "filtered": {  
> "query": { "match": {"field\_foo": "some search terms" }},  
> "filter": { "term": { "\_type": "type\_one" }}  
> }  
> }  
> ]  
> }  
> }  
> }

I meant:

"query": {  
"dis\_max": {  
"queries": [  
{  
"filtered": {  
"query": { "match": {"field\_foo": "some search terms" }},  
"filter": { "term": { "\_type": "type\_one" }}  
}  
},  
{  
"filtered": {  
"query": { "match": {"field\_foo": "some search terms" }},  
"filter": { "term": { "\_type": "type\_two" }}  
}  
}  
]  
}  
}  
}

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAPt3XKSSEg\_c60q5ZcsPZAXS2-boAdouHe3w8rNSzC5OitKyuQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPt3XKSSEg_c60q5ZcsPZAXS2-boAdouHe3w8rNSzC5OitKyuQ%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![golchhamohit](https://avatars.discourse-cdn.com/v4/letter/g/e19adc/32.png) [@golchhamohit](https://discuss.elastic.co/u/golchhamohit)\
**Post date:** [March 18, 2014, 8:56am UTC](https://discuss.elastic.co/t/mechanism-of-internal-search-with-multiple-indices/16399/4 "2014-03-18T08:56:07Z")

</div>

- deleted -

---

<div class="post-metadata">

**Author:** ![golchhamohit](https://avatars.discourse-cdn.com/v4/letter/g/e19adc/32.png) [@golchhamohit](https://discuss.elastic.co/u/golchhamohit)\
**Post date:** [March 18, 2014, 9:14am UTC](https://discuss.elastic.co/t/mechanism-of-internal-search-with-multiple-indices/16399/5 "2014-03-18T09:14:35Z")

</div>

@ Clinton : Many Thanks for explaining clearly how the query modifies  
itself when a query is given with multiple indices and types.

My another doubt is that which I represent here. I have multiple indices  
(assume 5 - i1,i2,i3,i4,i5), multiple types(5 in each index, so total 25  
types - t1,t2...,t25) , some field (called "field\_foo" which is present in  
all documents of all indices) and there are 100 documents in each  
index(d1,d2,..,d100). So total there are 5\*100 = 500 documents totally.

Now, my search query contains 2 indices(i2,i4) and  
types(t2,t5,t7,t13,t17,t23) and to search for a particular list of values  
["value\_field\_foo1",value\_field\_foo2",value\_field\_foo3"] in field  
"field\_foo" (where actually value\_field\_foo1 is present in index2,  
value\_field\_foo2 is present in index2 and index4, assume this, but in real  
time we would not be aware which value is present in which field). My  
question is will all the values(value\_field\_foo1, value\_field\_foo2,etc) be  
checked its presence in the documents of all indexes , or is there some  
mechanism by which it determines that value is not present in that index  
and hence it avoids the search in that index thereby saving time. ?

Sorry for being verbose.

Thanks in advance.

> 

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/d3012496-dd6d-4440-938d-d1b99d527666%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d3012496-dd6d-4440-938d-d1b99d527666%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\
**Post date:** [March 18, 2014, 5:15pm UTC](https://discuss.elastic.co/t/mechanism-of-internal-search-with-multiple-indices/16399/6 "2014-03-18T17:15:57Z")

</div>

If the field doesn't exist in the mapping, then the index is not searched.

clint

On 18 March 2014 09:56, golchhamohit [golchhamohit@gmail.com](mailto:golchhamohit@gmail.com) wrote:

> Thanks for explaining clearly how the query modifies itself when a query is  
> given with multiple indices and types.
> 
> My another doubt is that which I represent here. I have multiple indices  
> (assume 5 - i1,i2,i3,i4,i5), multiple types(5 in each index, so total 25  
> types - t1,t2...,t25) , some field (called "field\_foo" which is present in  
> all documents of all indices) and there are 100 documents in each  
> index(d1,d2,..,d100). So total there are 5\*100 = 500 documents totally.
> 
> Now, my search query contains 2 indices(i2,i4) and  
> types(t2,t5,t7,t13,t17,t23) and to search for a particular list of values  
> ["value\_field\_foo1",value\_field\_foo2",value\_field\_foo3"] in field  
> "field\_foo" (where actually value\_field\_foo1 is present in index2,  
> value\_field\_foo2 is present in index2 and index4, assume this, but in real  
> time we would not be aware which value is present in which field). My  
> question is will all the values(value\_field\_foo1, value\_field\_foo2,etc) be  
> checked its presence in the documents of all indexes , or is there some  
> mechanism by which it determines that value is not present in that index  
> and  
> hence it avoids the search in that index thereby saving time. ?
> 
> Sorry for being verbose.
> 
> Thanks in advance.
> 
> --  
> View this message in context:  
> [http://elasticsearch-users.115913.n3.nabble.com/Mechanism-of-internal-search-with-multiple-indices-tp4051988p4052080.html](http://elasticsearch-users.115913.n3.nabble.com/Mechanism-of-internal-search-with-multiple-indices-tp4051988p4052080.html)  
> Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com).
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/1395132968292-4052080.post%40n3.nabble.com](https://groups.google.com/d/msgid/elasticsearch/1395132968292-4052080.post%40n3.nabble.com)  
> .  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAPt3XKQV2d6EXmW7ypSzSN-yaUFyrjYG%2BZ9%3Dhc5pik\_9Nm4Riw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPt3XKQV2d6EXmW7ypSzSN-yaUFyrjYG%2BZ9%3Dhc5pik_9Nm4Riw%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:42am UTC](https://discuss.elastic.co/t/mechanism-of-internal-search-with-multiple-indices/16399/7 "2017-07-06T01:42:28Z")

</div>