# Nested range query is slow, slower with \_cache, how to debug?

**URL:** <https://discuss.elastic.co/t/nested-range-query-is-slow-slower-with--cache-how-to-debug/13623>\
**Category:** Elasticsearch\
**Created:** [September 16, 2013, 3:09pm UTC](https://discuss.elastic.co/t/nested-range-query-is-slow-slower-with--cache-how-to-debug/13623 "2013-09-16T15:09:48Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Damien\_Alexandre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/damien_alexandre/32/21566_2.png) [@Damien\_Alexandre](https://discuss.elastic.co/u/Damien_Alexandre)\
**Post date:** [September 16, 2013, 3:09pm UTC](https://discuss.elastic.co/t/nested-range-query-is-slow-slower-with--cache-how-to-debug/13623/1 "2013-09-16T15:09:48Z")

</div>

Hi everyone,

ES 0.90.3, 5 shards.

I run an index with a nested field,  
I have like 6 billions documents, and I run a query like this:

{  
"query": {  
"filtered": {  
_"query": { "match\_all": {} },_  
"filter": {  
"nested": {  
"path": "rights.display",  
"query": {  
"bool": {  
"must": [  
{  
"field": {  
"rights.display.zones": {  
"query": "FR"  
}  
}  
},  
{  
"range": {  
"rights.display.end\_date": {  
"gte": "2013-09-16"  
}  
}  
},  
{  
"range": {  
"rights.display.start\_date": {  
"lte": "2013-09-16"  
}  
}  
}  
]  
}  
}  
}  
}  
}  
}  
}

If I remove the two range part, queries perform really fast (_5/6ms_),  
but with them it took _1 second_ average.

So I have tried what the documentation tell about Nested Filter[http://www.elasticsearch.org/guide/reference/query-dsl/nested-filter/](http://www.elasticsearch.org/guide/reference/query-dsl/nested-filter/)  
:

_"\_cache" : true, "\_name": "testing\_FR"_

With this \_cache rule, results are slower: _2 to 3 seconds_!!

I have no idea how to debug this,  
here is a quick gist: [https://gist.github.com/damienalexandre/6581850](https://gist.github.com/damienalexandre/6581850)  
But without massive datas the difference between cached and not cached is  
not as clear as what I get.

I can see two issues here:

- my range query are slow, I guess this is the cost of doing a date range  
accross billions docs ;
- my nested filter is not cached, trying to set the cache make the query  
slower.

I'm looking for advice and tips on how to debug this,  
maybe it's a bug, but before creating an issue on github I think another  
pair of eyes can't hurt.

PS : I have also tried to set the filter to an alias - same perf issue.

Thx a lot,  
Damien

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![brian\_yoder](https://avatars.discourse-cdn.com/v4/letter/b/f1d935/32.png) [@brian\_yoder](https://discuss.elastic.co/u/brian_yoder)\
**Post date:** [September 16, 2013, 4:34pm UTC](https://discuss.elastic.co/t/nested-range-query-is-slow-slower-with--cache-how-to-debug/13623/2 "2013-09-16T16:34:00Z")

</div>

Hi Damien,

This is perhaps naive of me, but what I've seen work well across 100M  
documents (about 1/20 the number of documents you mentioned), the best  
range performance is when the range query is wrapped inside a bool query.  
For example (with the actual gn and sn query values changed to protect the  
innocent):

{  
"from" : 0,  
"size" : 20,  
"timeout" : 60000,  
"query" : {  
"bool" : {  
"must" : [ {  
"match" : {  
"gn" : {  
"query" : "aurelio",  
"type" : "boolean"  
}  
}  
}, {  
"match" : {  
"sn" : {  
"query" : "phzee",  
"type" : "boolean"  
}  
}  
}, {  
"range" : {  
"hn" : {  
"from" : 1000,  
"to" : 2000,  
"include\_lower" : true,  
"include\_upper" : true  
}  
}  
} ]  
}  
},  
"version" : true,  
"explain" : false,  
"fields" : ["\_ttl", "\_source"]  
}

This query took 3.4 seconds to return 5 documents out of 34 when the  
numeric range was omitted. But it did get much faster on subsequent  
queries, down to 100ms or less.

I hope this helps!

P.S. My client actually builds the queries in Java, and then can emit them  
as JSON for debugging and explanatory reasons.

Brian

On Monday, September 16, 2013 11:09:48 AM UTC-4, Damien Alexandre wrote:

> Hi everyone,
> 
> ES 0.90.3, 5 shards.
> 
> I run an index with a nested field,  
> I have like 6 billions documents, and I run a query like this:
> 
> {  
> "query": {  
> "filtered": {  
> _"query": { "match\_all": {} },_  
> "filter": {  
> "nested": {  
> "path": "rights.display",  
> "query": {  
> "bool": {  
> "must": [  
> {  
> "field": {  
> "rights.display.zones": {  
> "query": "FR"  
> }  
> }  
> },  
> {  
> "range": {  
> "rights.display.end\_date": {  
> "gte": "2013-09-16"  
> }  
> }  
> },  
> {  
> "range": {  
> "rights.display.start\_date": {  
> "lte": "2013-09-16"  
> }  
> }  
> }  
> ]  
> }  
> }  
> }  
> }  
> }  
> }  
> }

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![mvg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvg/32/98890_2.png) [@mvg](https://discuss.elastic.co/u/mvg)\
**Post date:** [September 16, 2013, 7:58pm UTC](https://discuss.elastic.co/t/nested-range-query-is-slow-slower-with--cache-how-to-debug/13623/3 "2013-09-16T19:58:59Z")

</div>

Hi Damien,

I would change the range query into range filter, then each range filter be  
cached on its own by default:

> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.

The range query doesn't cache at all on its own. If you wrap a filtered  
query as inner query in the nested filter and put the range filters in the  
filter part and the fields query in the query part then I expect a faster  
execution time:

```auto
{
  "query": {
    "filtered": {
      "query": { "match_all": {} },
      "filter": {
        "nested": {
          "path": "rights.display",
          "query": {
            "filtered": {
               "query": {
                    "field": {
                        "rights.display.zones": {
                          "query": "FR"
                        }
                    }
               },
               "filter": {
                   "bool": {
                       "must": [
                        {
                          "range": {
                            "rights.display.end_date": {
                              "gte": "2013-09-16"
                            }
                          }
                        },
                        {
                          "range": {
                            "rights.display.start_date": {
                              "lte": "2013-09-16"
                            }
                          }
                        }
                        ]
                   }
               }
            }
          }
        }
      }
    }
  }
}

```

The first time the range filters are executed these execution time is  
similar than the range query, but any subsequent search request should be  
much faster.  
Also I see that you're filtering on a day precession, are you also indexing  
the dates into the same precession? If not then I expect the range filter  
(and query) to execute better if you do this.

Also caching the nested filter doesn't really help, if one element in the  
nested filter changes than the cached entry can't be reused, and the nested  
filter needs to be completely re-executed.

Let me know if these tips helped out.

On 16 September 2013 18:34, InquiringMind [brian.from.fl@gmail.com](mailto:brian.from.fl@gmail.com) wrote:

> Hi Damien,
> 
> This is perhaps naive of me, but what I've seen work well across 100M  
> documents (about 1/20 the number of documents you mentioned), the best  
> range performance is when the range query is wrapped inside a bool query.  
> For example (with the actual gn and sn query values changed to protect the  
> innocent):
> 
> {  
> "from" : 0,  
> "size" : 20,  
> "timeout" : 60000,  
> "query" : {  
> "bool" : {  
> "must" : [ {  
> "match" : {  
> "gn" : {  
> "query" : "aurelio",  
> "type" : "boolean"  
> }  
> }  
> }, {  
> "match" : {  
> "sn" : {  
> "query" : "phzee",  
> "type" : "boolean"  
> }  
> }  
> }, {  
> "range" : {  
> "hn" : {  
> "from" : 1000,  
> "to" : 2000,  
> "include\_lower" : true,  
> "include\_upper" : true  
> }  
> }  
> } ]  
> }  
> },  
> "version" : true,  
> "explain" : false,  
> "fields" : ["\_ttl", "\_source"]  
> }
> 
> This query took 3.4 seconds to return 5 documents out of 34 when the  
> numeric range was omitted. But it did get much faster on subsequent  
> queries, down to 100ms or less.
> 
> I hope this helps!
> 
> P.S. My client actually builds the queries in Java, and then can emit them  
> as JSON for debugging and explanatory reasons.
> 
> Brian
> 
> On Monday, September 16, 2013 11:09:48 AM UTC-4, Damien Alexandre wrote:
> 
> > Hi everyone,
> > 
> > ES 0.90.3, 5 shards.
> > 
> > I run an index with a nested field,  
> > I have like 6 billions documents, and I run a query like this:
> > 
> > {  
> > "query": {  
> > "filtered": {  
> > _"query": { "match\_all": {} },_  
> > "filter": {  
> > "nested": {  
> > "path": "rights.display",  
> > "query": {  
> > "bool": {  
> > "must": [  
> > {  
> > "field": {  
> > "rights.display.zones": {  
> > "query": "FR"  
> > }  
> > }  
> > },  
> > {  
> > "range": {  
> > "rights.display.end\_date": {  
> > "gte": "2013-09-16"  
> > }  
> > }  
> > },  
> > {  
> > "range": {  
> > "rights.display.start\_date": {  
> > "lte": "2013-09-16"  
> > }  
> > }  
> > }  
> > ]  
> > }  
> > }  
> > }  
> > }  
> > }  
> > }  
> > }
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
Met vriendelijke groet,

Martijn van Groningen

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Damien\_Alexandre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/damien_alexandre/32/21566_2.png) [@Damien\_Alexandre](https://discuss.elastic.co/u/Damien_Alexandre)\
**Post date:** [September 17, 2013, 8:22am UTC](https://discuss.elastic.co/t/nested-range-query-is-slow-slower-with--cache-how-to-debug/13623/4 "2013-09-17T08:22:31Z")

</div>

Hi,

using Range filter instead of Range query works very well! I dropped from  
300ms to 10ms on a lot of my queries!  
Still, I think it's strange that the Nested Filter cache does not work  
better than the Range filter one's - looks odd to me, but anyway :]

About the date, yes they are indexed with a day precision, like in my  
queries - so it's kind of fast now,  
I apply sort, filters, queries... on billions of documents with nested  
filed and now I get my results in 10ms: that's awesome ❤

Thanks Martijn & Brian!

[http://gph.is/XL6HqD](http://gph.is/XL6HqD)

Damien.

On Monday, September 16, 2013 9:58:59 PM UTC+2, Martijn v Groningen wrote:

> Hi Damien,
> 
> I would change the range query into range filter, then each range filter  
> be cached on its own by default:  
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/query-dsl/range-filter/)
> 
> The range query doesn't cache at all on its own. If you wrap a filtered  
> query as inner query in the nested filter and put the range filters in the  
> filter part and the fields query in the query part then I expect a faster  
> execution time:
> 
> ```auto
> {
> "query": {
> "filtered": {
> "query": { "match_all": {} },
> "filter": {
> "nested": {
> "path": "rights.display",
> "query": {
> "filtered": {
> "query": {
> "field": {
> "rights.display.zones": {
> "query": "FR"
> }
> }
> },
> "filter": {
> "bool": {
> "must": [
> {
> "range": {
> "rights.display.end_date": {
> "gte": "2013-09-16"
> }
> }
> },
> {
> "range": {
> "rights.display.start_date": {
> "lte": "2013-09-16"
> }
> }
> }
> ]
> }
> }
> }
> }
> }
> }
> }
> }
> }
> 
> ```
> 
> The first time the range filters are executed these execution time is  
> similar than the range query, but any subsequent search request should be  
> much faster.  
> Also I see that you're filtering on a day precession, are you also  
> indexing the dates into the same precession? If not then I expect the range  
> filter (and query) to execute better if you do this.
> 
> Also caching the nested filter doesn't really help, if one element in the  
> nested filter changes than the cached entry can't be reused, and the nested  
> filter needs to be completely re-executed.
> 
> Let me know if these tips helped out.
> 
> On 16 September 2013 18:34, InquiringMind \<[brian....@gmail.com](mailto:brian....@gmail.com)\<javascript:\>
> 
> > wrote:
> 
> > Hi Damien,
> > 
> > This is perhaps naive of me, but what I've seen work well across 100M  
> > documents (about 1/20 the number of documents you mentioned), the best  
> > range performance is when the range query is wrapped inside a bool query.  
> > For example (with the actual gn and sn query values changed to protect the  
> > innocent):
> > 
> > {  
> > "from" : 0,  
> > "size" : 20,  
> > "timeout" : 60000,  
> > "query" : {  
> > "bool" : {  
> > "must" : [ {  
> > "match" : {  
> > "gn" : {  
> > "query" : "aurelio",  
> > "type" : "boolean"  
> > }  
> > }  
> > }, {  
> > "match" : {  
> > "sn" : {  
> > "query" : "phzee",  
> > "type" : "boolean"  
> > }  
> > }  
> > }, {  
> > "range" : {  
> > "hn" : {  
> > "from" : 1000,  
> > "to" : 2000,  
> > "include\_lower" : true,  
> > "include\_upper" : true  
> > }  
> > }  
> > } ]  
> > }  
> > },  
> > "version" : true,  
> > "explain" : false,  
> > "fields" : ["\_ttl", "\_source"]  
> > }
> > 
> > This query took 3.4 seconds to return 5 documents out of 34 when the  
> > numeric range was omitted. But it did get much faster on subsequent  
> > queries, down to 100ms or less.
> > 
> > I hope this helps!
> > 
> > P.S. My client actually builds the queries in Java, and then can emit  
> > them as JSON for debugging and explanatory reasons.
> > 
> > Brian
> > 
> > On Monday, September 16, 2013 11:09:48 AM UTC-4, Damien Alexandre wrote:
> > 
> > > Hi everyone,
> > > 
> > > ES 0.90.3, 5 shards.
> > > 
> > > I run an index with a nested field,  
> > > I have like 6 billions documents, and I run a query like this:
> > > 
> > > {  
> > > "query": {  
> > > "filtered": {  
> > > _"query": { "match\_all": {} },_  
> > > "filter": {  
> > > "nested": {  
> > > "path": "rights.display",  
> > > "query": {  
> > > "bool": {  
> > > "must": [  
> > > {  
> > > "field": {  
> > > "rights.display.zones": {  
> > > "query": "FR"  
> > > }  
> > > }  
> > > },  
> > > {  
> > > "range": {  
> > > "rights.display.end\_date": {  
> > > "gte": "2013-09-16"  
> > > }  
> > > }  
> > > },  
> > > {  
> > > "range": {  
> > > "rights.display.start\_date": {  
> > > "lte": "2013-09-16"  
> > > }  
> > > }  
> > > }  
> > > ]  
> > > }  
> > > }  
> > > }  
> > > }  
> > > }  
> > > }  
> > > }
> > > 
> > > --  
> > > You received this message because you are subscribed to the Google Groups  
> > > "elasticsearch" group.  
> > > To unsubscribe from this group and stop receiving emails from it, send an  
> > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \<javascript:\>.  
> > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> 
> --  
> Met vriendelijke groet,
> 
> Martijn van Groningen

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\
**Post date:** [September 17, 2013, 9:52pm UTC](https://discuss.elastic.co/t/nested-range-query-is-slow-slower-with--cache-how-to-debug/13623/5 "2013-09-17T21:52:30Z")

</div>

Don't range filter work better with and/or/not filter and not inside bool  
filters due to bitset caching? Never profiled myself.

--  
Ivan

On Mon, Sep 16, 2013 at 12:58 PM, Martijn v Groningen \<  
[martijn.v.groningen@gmail.com](mailto:martijn.v.groningen@gmail.com)\> wrote:

> Hi Damien,
> 
> I would change the range query into range filter, then each range filter  
> be cached on its own by default:  
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/query-dsl/range-filter/)
> 
> The range query doesn't cache at all on its own. If you wrap a filtered  
> query as inner query in the nested filter and put the range filters in the  
> filter part and the fields query in the query part then I expect a faster  
> execution time:
> 
> ```auto
> {
> "query": {
> "filtered": {
> "query": { "match_all": {} },
> "filter": {
> "nested": {
> "path": "rights.display",
> "query": {
> "filtered": {
> "query": {
> "field": {
> "rights.display.zones": {
> "query": "FR"
> }
> }
> },
> "filter": {
> "bool": {
> "must": [
> {
> "range": {
> "rights.display.end_date": {
> "gte": "2013-09-16"
> }
> }
> },
> {
> "range": {
> "rights.display.start_date": {
> "lte": "2013-09-16"
> }
> }
> }
> ]
> }
> }
> }
> }
> }
> }
> }
> }
> }
> 
> ```
> 
> The first time the range filters are executed these execution time is  
> similar than the range query, but any subsequent search request should be  
> much faster.  
> Also I see that you're filtering on a day precession, are you also  
> indexing the dates into the same precession? If not then I expect the range  
> filter (and query) to execute better if you do this.
> 
> Also caching the nested filter doesn't really help, if one element in the  
> nested filter changes than the cached entry can't be reused, and the nested  
> filter needs to be completely re-executed.
> 
> Let me know if these tips helped out.
> 
> On 16 September 2013 18:34, InquiringMind [brian.from.fl@gmail.com](mailto:brian.from.fl@gmail.com) wrote:
> 
> > Hi Damien,
> > 
> > This is perhaps naive of me, but what I've seen work well across 100M  
> > documents (about 1/20 the number of documents you mentioned), the best  
> > range performance is when the range query is wrapped inside a bool query.  
> > For example (with the actual gn and sn query values changed to protect the  
> > innocent):
> > 
> > {  
> > "from" : 0,  
> > "size" : 20,  
> > "timeout" : 60000,  
> > "query" : {  
> > "bool" : {  
> > "must" : [ {  
> > "match" : {  
> > "gn" : {  
> > "query" : "aurelio",  
> > "type" : "boolean"  
> > }  
> > }  
> > }, {  
> > "match" : {  
> > "sn" : {  
> > "query" : "phzee",  
> > "type" : "boolean"  
> > }  
> > }  
> > }, {  
> > "range" : {  
> > "hn" : {  
> > "from" : 1000,  
> > "to" : 2000,  
> > "include\_lower" : true,  
> > "include\_upper" : true  
> > }  
> > }  
> > } ]  
> > }  
> > },  
> > "version" : true,  
> > "explain" : false,  
> > "fields" : ["\_ttl", "\_source"]  
> > }
> > 
> > This query took 3.4 seconds to return 5 documents out of 34 when the  
> > numeric range was omitted. But it did get much faster on subsequent  
> > queries, down to 100ms or less.
> > 
> > I hope this helps!
> > 
> > P.S. My client actually builds the queries in Java, and then can emit  
> > them as JSON for debugging and explanatory reasons.
> > 
> > Brian
> > 
> > On Monday, September 16, 2013 11:09:48 AM UTC-4, Damien Alexandre wrote:
> > 
> > > Hi everyone,
> > > 
> > > ES 0.90.3, 5 shards.
> > > 
> > > I run an index with a nested field,  
> > > I have like 6 billions documents, and I run a query like this:
> > > 
> > > {  
> > > "query": {  
> > > "filtered": {  
> > > _"query": { "match\_all": {} },_  
> > > "filter": {  
> > > "nested": {  
> > > "path": "rights.display",  
> > > "query": {  
> > > "bool": {  
> > > "must": [  
> > > {  
> > > "field": {  
> > > "rights.display.zones": {  
> > > "query": "FR"  
> > > }  
> > > }  
> > > },  
> > > {  
> > > "range": {  
> > > "rights.display.end\_date": {  
> > > "gte": "2013-09-16"  
> > > }  
> > > }  
> > > },  
> > > {  
> > > "range": {  
> > > "rights.display.start\_date": {  
> > > "lte": "2013-09-16"  
> > > }  
> > > }  
> > > }  
> > > ]  
> > > }  
> > > }  
> > > }  
> > > }  
> > > }  
> > > }  
> > > }
> > > 
> > > --  
> > > You received this message because you are subscribed to the Google Groups  
> > > "elasticsearch" group.  
> > > To unsubscribe from this group and stop receiving emails from it, send an  
> > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> 
> --  
> Met vriendelijke groet,
> 
> Martijn van Groningen
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![mvg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvg/32/98890_2.png) [@mvg](https://discuss.elastic.co/u/mvg)\
**Post date:** [September 18, 2013, 8:25am UTC](https://discuss.elastic.co/t/nested-range-query-is-slow-slower-with--cache-how-to-debug/13623/6 "2013-09-18T08:25:27Z")

</div>

The `range` filter should work best inside a bool filter. The  
`numeric_range` should work best inside an and/or filter, but only when it  
isn't cached (by default this filter is never cached).

On 17 September 2013 23:52, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote:

> Don't range filter work better with and/or/not filter and not inside bool  
> filters due to bitset caching? Never profiled myself.
> 
> --  
> Ivan
> 
> On Mon, Sep 16, 2013 at 12:58 PM, Martijn v Groningen \<  
> [martijn.v.groningen@gmail.com](mailto:martijn.v.groningen@gmail.com)\> wrote:
> 
> > Hi Damien,
> > 
> > I would change the range query into range filter, then each range filter  
> > be cached on its own by default:  
> > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/query-dsl/range-filter/)
> > 
> > The range query doesn't cache at all on its own. If you wrap a filtered  
> > query as inner query in the nested filter and put the range filters in the  
> > filter part and the fields query in the query part then I expect a faster  
> > execution time:
> > 
> > ```auto
> > {
> > "query": {
> > "filtered": {
> > "query": { "match_all": {} },
> > "filter": {
> > "nested": {
> > "path": "rights.display",
> > "query": {
> > "filtered": {
> > "query": {
> > "field": {
> > "rights.display.zones": {
> > "query": "FR"
> > }
> > }
> > },
> > "filter": {
> > "bool": {
> > "must": [
> > {
> > "range": {
> > "rights.display.end_date": {
> > "gte": "2013-09-16"
> > }
> > }
> > },
> > {
> > "range": {
> > "rights.display.start_date": {
> > "lte": "2013-09-16"
> > }
> > }
> > }
> > ]
> > }
> > }
> > }
> > }
> > }
> > }
> > }
> > }
> > }
> > 
> > ```
> > 
> > The first time the range filters are executed these execution time is  
> > similar than the range query, but any subsequent search request should be  
> > much faster.  
> > Also I see that you're filtering on a day precession, are you also  
> > indexing the dates into the same precession? If not then I expect the range  
> > filter (and query) to execute better if you do this.
> > 
> > Also caching the nested filter doesn't really help, if one element in the  
> > nested filter changes than the cached entry can't be reused, and the nested  
> > filter needs to be completely re-executed.
> > 
> > Let me know if these tips helped out.
> > 
> > On 16 September 2013 18:34, InquiringMind [brian.from.fl@gmail.com](mailto:brian.from.fl@gmail.com)wrote:
> > 
> > > Hi Damien,
> > > 
> > > This is perhaps naive of me, but what I've seen work well across 100M  
> > > documents (about 1/20 the number of documents you mentioned), the best  
> > > range performance is when the range query is wrapped inside a bool query.  
> > > For example (with the actual gn and sn query values changed to protect the  
> > > innocent):
> > > 
> > > {  
> > > "from" : 0,  
> > > "size" : 20,  
> > > "timeout" : 60000,  
> > > "query" : {  
> > > "bool" : {  
> > > "must" : [ {  
> > > "match" : {  
> > > "gn" : {  
> > > "query" : "aurelio",  
> > > "type" : "boolean"  
> > > }  
> > > }  
> > > }, {  
> > > "match" : {  
> > > "sn" : {  
> > > "query" : "phzee",  
> > > "type" : "boolean"  
> > > }  
> > > }  
> > > }, {  
> > > "range" : {  
> > > "hn" : {  
> > > "from" : 1000,  
> > > "to" : 2000,  
> > > "include\_lower" : true,  
> > > "include\_upper" : true  
> > > }  
> > > }  
> > > } ]  
> > > }  
> > > },  
> > > "version" : true,  
> > > "explain" : false,  
> > > "fields" : ["\_ttl", "\_source"]  
> > > }
> > > 
> > > This query took 3.4 seconds to return 5 documents out of 34 when the  
> > > numeric range was omitted. But it did get much faster on subsequent  
> > > queries, down to 100ms or less.
> > > 
> > > I hope this helps!
> > > 
> > > P.S. My client actually builds the queries in Java, and then can emit  
> > > them as JSON for debugging and explanatory reasons.
> > > 
> > > Brian
> > > 
> > > On Monday, September 16, 2013 11:09:48 AM UTC-4, Damien Alexandre wrote:
> > > 
> > > > Hi everyone,
> > > > 
> > > > ES 0.90.3, 5 shards.
> > > > 
> > > > I run an index with a nested field,  
> > > > I have like 6 billions documents, and I run a query like this:
> > > > 
> > > > {  
> > > > "query": {  
> > > > "filtered": {  
> > > > _"query": { "match\_all": {} },_  
> > > > "filter": {  
> > > > "nested": {  
> > > > "path": "rights.display",  
> > > > "query": {  
> > > > "bool": {  
> > > > "must": [  
> > > > {  
> > > > "field": {  
> > > > "rights.display.zones": {  
> > > > "query": "FR"  
> > > > }  
> > > > }  
> > > > },  
> > > > {  
> > > > "range": {  
> > > > "rights.display.end\_date": {  
> > > > "gte": "2013-09-16"  
> > > > }  
> > > > }  
> > > > },  
> > > > {  
> > > > "range": {  
> > > > "rights.display.start\_date": {  
> > > > "lte": "2013-09-16"  
> > > > }  
> > > > }  
> > > > }  
> > > > ]  
> > > > }  
> > > > }  
> > > > }  
> > > > }  
> > > > }  
> > > > }  
> > > > }
> > > > 
> > > > --  
> > > > You received this message because you are subscribed to the Google  
> > > > Groups "elasticsearch" group.  
> > > > To unsubscribe from this group and stop receiving emails from it, send  
> > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > 
> > --  
> > Met vriendelijke groet,
> > 
> > Martijn van Groningen
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
Met vriendelijke groet,

Martijn van Groningen

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 2:15am UTC](https://discuss.elastic.co/t/nested-range-query-is-slow-slower-with--cache-how-to-debug/13623/7 "2017-07-06T02:15:48Z")

</div>


