# Bad response time when using date ranges

**URL:** <https://discuss.elastic.co/t/bad-response-time-when-using-date-ranges/13366>\
**Category:** Elasticsearch\
**Created:** [August 28, 2013, 2:01pm UTC](https://discuss.elastic.co/t/bad-response-time-when-using-date-ranges/13366 "2013-08-28T14:01:39Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Matias\_Waisgold](https://avatars.discourse-cdn.com/v4/letter/m/8e8cbc/32.png) [@Matias\_Waisgold](https://discuss.elastic.co/u/Matias_Waisgold)\
**Post date:** [August 28, 2013, 2:01pm UTC](https://discuss.elastic.co/t/bad-response-time-when-using-date-ranges/13366/1 "2013-08-28T14:01:39Z")

</div>

I'm having a collection with 8M documents. I want to retrieve last 20 days  
of a "seller\_id", i'm doing this query:

{  
"query": {  
"term": {  
"seller\_id": 83013710  
}  
},  
"filter": {  
"range": {  
"date\_created": {  
"from": "now-20d"  
}  
}  
},  
"size": 50  
}

The problem is that it's taking about 200ms for retrieve only 200 results.  
And when i search only by seller\_id it takes 3ms. How can I improve this  
query? Any best practice on date fields? (It's in dateTime format and with  
the default precision\_step)

Kind regards.  
Matías

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![mattweber](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mattweber/32/44940_2.png) [@mattweber](https://discuss.elastic.co/u/mattweber)\
**Post date:** [August 28, 2013, 2:42pm UTC](https://discuss.elastic.co/t/bad-response-time-when-using-date-ranges/13366/2 "2013-08-28T14:42:24Z")

</div>

Use a filtered query vs. the outer filter and round your "now" by day  
(now/d-20d) so filter can get cached and reused.... If you don't need  
scoring, use a constant score query with Boolean filter that has  
the term+range filter as must clauses. That should give better  
performance.

On Wednesday, August 28, 2013, Matías Waisgold wrote:

> I'm having a collection with 8M documents. I want to retrieve last 20 days  
> of a "seller\_id", i'm doing this query:
> 
> {  
> "query": {  
> "term": {  
> "seller\_id": 83013710  
> }  
> },  
> "filter": {  
> "range": {  
> "date\_created": {  
> "from": "now-20d"  
> }  
> }  
> },  
> "size": 50  
> }
> 
> The problem is that it's taking about 200ms for retrieve only 200 results.  
> And when i search only by seller\_id it takes 3ms. How can I improve this  
> query? Any best practice on date fields? (It's in dateTime format and with  
> the default precision\_step)
> 
> Kind regards.  
> Matías
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) \<javascript:\_e({},  
> 'cvml', 'elasticsearch%2Bunsubscribe@googlegroups.com');\>.  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
Thanks,  
Matt Weber

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Matias\_Waisgold](https://avatars.discourse-cdn.com/v4/letter/m/8e8cbc/32.png) [@Matias\_Waisgold](https://discuss.elastic.co/u/Matias_Waisgold)\
**Post date:** [August 28, 2013, 3:38pm UTC](https://discuss.elastic.co/t/bad-response-time-when-using-date-ranges/13366/3 "2013-08-28T15:38:57Z")

</div>

WOW, that's awesome. Just to understand it better, why does this rounding  
improves performance?

2013/8/28 Matt Weber [matt@mattweber.org](mailto:matt@mattweber.org)

> Use a filtered query vs. the outer filter and round your "now" by day  
> (now/d-20d) so filter can get cached and reused.... If you don't need  
> scoring, use a constant score query with Boolean filter that has  
> the term+range filter as must clauses. That should give better  
> performance.
> 
> On Wednesday, August 28, 2013, Matías Waisgold wrote:
> 
> > I'm having a collection with 8M documents. I want to retrieve last 20  
> > days of a "seller\_id", i'm doing this query:
> > 
> > {  
> > "query": {  
> > "term": {  
> > "seller\_id": 83013710  
> > }  
> > },  
> > "filter": {  
> > "range": {  
> > "date\_created": {  
> > "from": "now-20d"  
> > }  
> > }  
> > },  
> > "size": 50  
> > }
> > 
> > The problem is that it's taking about 200ms for retrieve only 200  
> > results. And when i search only by seller\_id it takes 3ms. How can I  
> > improve this query? Any best practice on date fields? (It's in dateTime  
> > format and with the default precision\_step)
> > 
> > Kind regards.  
> > Matías
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> 
> --  
> Thanks,  
> Matt Weber
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![mattweber](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mattweber/32/44940_2.png) [@mattweber](https://discuss.elastic.co/u/mattweber)\
**Post date:** [August 28, 2013, 3:50pm UTC](https://discuss.elastic.co/t/bad-response-time-when-using-date-ranges/13366/4 "2013-08-28T15:50:22Z")

</div>

Without rounding your "now" includes milliseconds so it will never be able  
to be reused from the cache (since the ms have changed). When you round,  
the filter has better chances of being reused resulting in huge performance  
improvements.

Thanks,  
Matt Weber

On Wed, Aug 28, 2013 at 8:38 AM, Matías Waisgold [mwaisgold@gmail.com](mailto:mwaisgold@gmail.com)wrote:

> WOW, that's awesome. Just to understand it better, why does this rounding  
> improves performance?
> 
> 2013/8/28 Matt Weber [matt@mattweber.org](mailto:matt@mattweber.org)
> 
> > Use a filtered query vs. the outer filter and round your "now" by day  
> > (now/d-20d) so filter can get cached and reused.... If you don't need  
> > scoring, use a constant score query with Boolean filter that has  
> > the term+range filter as must clauses. That should give better  
> > performance.
> > 
> > On Wednesday, August 28, 2013, Matías Waisgold wrote:
> > 
> > > I'm having a collection with 8M documents. I want to retrieve last 20  
> > > days of a "seller\_id", i'm doing this query:
> > > 
> > > {  
> > > "query": {  
> > > "term": {  
> > > "seller\_id": 83013710  
> > > }  
> > > },  
> > > "filter": {  
> > > "range": {  
> > > "date\_created": {  
> > > "from": "now-20d"  
> > > }  
> > > }  
> > > },  
> > > "size": 50  
> > > }
> > > 
> > > The problem is that it's taking about 200ms for retrieve only 200  
> > > results. And when i search only by seller\_id it takes 3ms. How can I  
> > > improve this query? Any best practice on date fields? (It's in dateTime  
> > > format and with the default precision\_step)
> > > 
> > > Kind regards.  
> > > Matías
> > > 
> > > --  
> > > You received this message because you are subscribed to the Google  
> > > Groups "elasticsearch" group.  
> > > To unsubscribe from this group and stop receiving emails from it, send  
> > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > 
> > --  
> > Thanks,  
> > Matt Weber
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Matias\_Waisgold](https://avatars.discourse-cdn.com/v4/letter/m/8e8cbc/32.png) [@Matias\_Waisgold](https://discuss.elastic.co/u/Matias_Waisgold)\
**Post date:** [August 29, 2013, 3:34am UTC](https://discuss.elastic.co/t/bad-response-time-when-using-date-ranges/13366/5 "2013-08-29T03:34:14Z")

</div>

Great, thank's!

2013/8/28 Matt Weber [matt.weber@gmail.com](mailto:matt.weber@gmail.com)

> Without rounding your "now" includes milliseconds so it will never be able  
> to be reused from the cache (since the ms have changed). When you round,  
> the filter has better chances of being reused resulting in huge performance  
> improvements.
> 
> Thanks,  
> Matt Weber
> 
> On Wed, Aug 28, 2013 at 8:38 AM, Matías Waisgold [mwaisgold@gmail.com](mailto:mwaisgold@gmail.com)wrote:
> 
> > WOW, that's awesome. Just to understand it better, why does this rounding  
> > improves performance?
> > 
> > 2013/8/28 Matt Weber [matt@mattweber.org](mailto:matt@mattweber.org)
> > 
> > > Use a filtered query vs. the outer filter and round your "now" by day  
> > > (now/d-20d) so filter can get cached and reused.... If you don't need  
> > > scoring, use a constant score query with Boolean filter that has  
> > > the term+range filter as must clauses. That should give better  
> > > performance.
> > > 
> > > On Wednesday, August 28, 2013, Matías Waisgold wrote:
> > > 
> > > > I'm having a collection with 8M documents. I want to retrieve last 20  
> > > > days of a "seller\_id", i'm doing this query:
> > > > 
> > > > {  
> > > > "query": {  
> > > > "term": {  
> > > > "seller\_id": 83013710  
> > > > }  
> > > > },  
> > > > "filter": {  
> > > > "range": {  
> > > > "date\_created": {  
> > > > "from": "now-20d"  
> > > > }  
> > > > }  
> > > > },  
> > > > "size": 50  
> > > > }
> > > > 
> > > > The problem is that it's taking about 200ms for retrieve only 200  
> > > > results. And when i search only by seller\_id it takes 3ms. How can I  
> > > > improve this query? Any best practice on date fields? (It's in dateTime  
> > > > format and with the default precision\_step)
> > > > 
> > > > Kind regards.  
> > > > Matías
> > > > 
> > > > --  
> > > > You received this message because you are subscribed to the Google  
> > > > Groups "elasticsearch" group.  
> > > > To unsubscribe from this group and stop receiving emails from it, send  
> > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > 
> > > --  
> > > Thanks,  
> > > Matt Weber
> > > 
> > > --  
> > > You received this message because you are subscribed to the Google  
> > > Groups "elasticsearch" group.  
> > > To unsubscribe from this group and stop receiving emails from it, send  
> > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [August 29, 2013, 5:26am UTC](https://discuss.elastic.co/t/bad-response-time-when-using-date-ranges/13366/6 "2013-08-29T05:26:08Z")

</div>

Just to add, a numeric filter range on a simple int or long field is very  
fast.

[http://www.elasticsearch.org/guide/reference/query-dsl/numeric-range-filter/](http://www.elasticsearch.org/guide/reference/query-dsl/numeric-range-filter/)

In many cases, when doing date ranges, you never need milliseconds (or even  
seconds), so you should discretize the time stamps to coarser int values  
and change the field type respectively.

Jörg

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 2:19am UTC](https://discuss.elastic.co/t/bad-response-time-when-using-date-ranges/13366/7 "2017-07-06T02:19:10Z")

</div>