# Script terms facet in elasticsearch 0.90.0 returning incorrect results **URL:** \ **Category:** Elasticsearch\ **Created:** [May 3, 2013, 6:54am UTC](https://discuss.elastic.co/t/script-terms-facet-in-elasticsearch-0-90-0-returning-incorrect-results/11795 "2013-05-03T06:54:22Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Eric\_Jain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eric_jain/32/834_2.png) [@Eric\_Jain](https://discuss.elastic.co/u/Eric_Jain)\ **Post date:** [May 3, 2013, 6:54am UTC](https://discuss.elastic.co/t/script-terms-facet-in-elasticsearch-0-90-0-returning-incorrect-results/11795/1 "2013-05-03T06:54:22Z")
[https://gist.github.com/ejain/5507570](https://gist.github.com/ejain/5507570) (0.20.6) vs [https://gist.github.com/ejain/5507571](https://gist.github.com/ejain/5507571) (0.90.0) Looks like time zone offsets in dates are now ignored, and dates are assumed to be in the default time zone (i.e. Pacific Time for me) when being accessed from a script? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Eric\_Jain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eric_jain/32/834_2.png) [@Eric\_Jain](https://discuss.elastic.co/u/Eric_Jain)\ **Post date:** [May 7, 2013, 8:24pm UTC](https://discuss.elastic.co/t/script-terms-facet-in-elasticsearch-0-90-0-returning-incorrect-results/11795/2 "2013-05-07T20:24:41Z")
For the record: Dates in scripts now appear to use the default time zone; previously they were always UTC. Not necessarily a bug, but a breaking change to be aware of... On Thursday, May 2, 2013 11:54:22 PM UTC-7, Eric Jain wrote: > [Example request and (correct) response when using a script terms facet in elasticsearch 0.20.6 (+ lang-javascript 1.2.0). · GitHub](https://gist.github.com/ejain/5507570) (0.20.6) vs > [Example request and (incorrect) response when using a script terms facet in elasticsearch 0.90.0 (+ lang-javascript 1.3.0). · GitHub](https://gist.github.com/ejain/5507571) (0.90.0) > > Looks like time zone offsets in dates are now ignored, and dates are > assumed to be in the default time zone (i.e. Pacific Time for me) when > being accessed from a script? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Marcin\_Dojwa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marcin_dojwa/32/2307_2.png) [@Marcin\_Dojwa](https://discuss.elastic.co/u/Marcin_Dojwa)\ **Post date:** [May 8, 2013, 12:37pm UTC](https://discuss.elastic.co/t/script-terms-facet-in-elasticsearch-0-90-0-returning-incorrect-results/11795/3 "2013-05-08T12:37:17Z")
Thanks for this one. What happens if you give '2012-11-30T15:00:00Z'? The given datetime is treated as UTC or default timezone too ? 2013/5/7 Eric Jain [eric.jain@gmail.com](mailto:eric.jain@gmail.com) > For the record: Dates in scripts now appear to use the default time zone; > previously they were always UTC. Not necessarily a bug, but a breaking > change to be aware of... > > On Thursday, May 2, 2013 11:54:22 PM UTC-7, Eric Jain wrote: > > > [https://gist.github.com/ejain/\*\*5507570](https://gist.github.com/ejain/**5507570)[https://gist.github.com/ejain/5507570](https://gist.github.com/ejain/5507570)(0.20.6) vs > > [https://gist.github.com/ejain/\*\*5507571](https://gist.github.com/ejain/**5507571)[https://gist.github.com/ejain/5507571](https://gist.github.com/ejain/5507571)(0.90.0) > > > > Looks like time zone offsets in dates are now ignored, and dates are > > assumed to be in the default time zone (i.e. Pacific Time for me) when > > being accessed from a script? > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Eric\_Jain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eric_jain/32/834_2.png) [@Eric\_Jain](https://discuss.elastic.co/u/Eric_Jain)\ **Post date:** [May 8, 2013, 4:29pm UTC](https://discuss.elastic.co/t/script-terms-facet-in-elasticsearch-0-90-0-returning-incorrect-results/11795/4 "2013-05-08T16:29:10Z")
On Wed, May 8, 2013 at 5:37 AM, Marcin Dojwa [m.dojwa@livechatinc.com](mailto:m.dojwa@livechatinc.com) wrote: > Thanks for this one. What happens if you give '2012-11-30T15:00:00Z'? The > given datetime is treated as UTC or default timezone too ? If your local timezone offset is e.g. -07:00, doc['timestamp'].date is now (0.90) '2012-11-30T08:00:00-07:00'. Previously (0.20), it would have been '2012-11-30T15:00:00Z'; not because elasticsearch was honoring the date's original timezone offset (I wish it did!), but because the `date` property in a script would be always be UTC. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Marcin\_Dojwa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marcin_dojwa/32/2307_2.png) [@Marcin\_Dojwa](https://discuss.elastic.co/u/Marcin_Dojwa)\ **Post date:** [May 9, 2013, 8:45am UTC](https://discuss.elastic.co/t/script-terms-facet-in-elasticsearch-0-90-0-returning-incorrect-results/11795/5 "2013-05-09T08:45:13Z")
For me it sounds like a bug. Did you report the issue here [Issues · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues) ? 2013/5/8 Eric Jain [eric.jain@gmail.com](mailto:eric.jain@gmail.com) > On Wed, May 8, 2013 at 5:37 AM, Marcin Dojwa [m.dojwa@livechatinc.com](mailto:m.dojwa@livechatinc.com) > wrote: > > > Thanks for this one. What happens if you give '2012-11-30T15:00:00Z'? The > > given datetime is treated as UTC or default timezone too ? > > If your local timezone offset is e.g. -07:00, doc['timestamp'].date is > now (0.90) '2012-11-30T08:00:00-07:00'. Previously (0.20), it would > have been '2012-11-30T15:00:00Z'; not because elasticsearch was > honoring the date's original timezone offset (I wish it did!), but > because the `date` property in a script would be always be UTC. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Eric\_Jain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eric_jain/32/834_2.png) [@Eric\_Jain](https://discuss.elastic.co/u/Eric_Jain)\ **Post date:** [May 9, 2013, 1:52pm UTC](https://discuss.elastic.co/t/script-terms-facet-in-elasticsearch-0-90-0-returning-incorrect-results/11795/6 "2013-05-09T13:52:27Z")
On Thu, May 9, 2013 at 1:45 AM, Marcin Dojwa [m.dojwa@livechatinc.com](mailto:m.dojwa@livechatinc.com) wrote: > For me it sounds like a bug. Did you report the issue here > [Issues · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues) ? I haven't report the issue yet; I was hoping to hear from someone at elasticsearch whether this change was intentional or not. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Marcin\_Dojwa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marcin_dojwa/32/2307_2.png) [@Marcin\_Dojwa](https://discuss.elastic.co/u/Marcin_Dojwa)\ **Post date:** [May 10, 2013, 5:38am UTC](https://discuss.elastic.co/t/script-terms-facet-in-elasticsearch-0-90-0-returning-incorrect-results/11795/7 "2013-05-10T05:38:49Z")
I suggest reporting an issue. If this is not an issue, you will get fast response there. Could you give a link to the issue here when you report it ? Solving this problem is very important for me too before I deploy 0.90 on production. Thanks. 2013/5/9 Eric Jain [eric.jain@gmail.com](mailto:eric.jain@gmail.com) > On Thu, May 9, 2013 at 1:45 AM, Marcin Dojwa [m.dojwa@livechatinc.com](mailto:m.dojwa@livechatinc.com) > wrote: > > > For me it sounds like a bug. Did you report the issue here > > [Issues · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues) ? > > I haven't report the issue yet; I was hoping to hear from someone at > elasticsearch whether this change was intentional or not. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![phill](https://avatars.discourse-cdn.com/v4/letter/p/779978/32.png) [@phill](https://discuss.elastic.co/u/phill)\ **Post date:** [May 10, 2013, 6:50pm UTC](https://discuss.elastic.co/t/script-terms-facet-in-elasticsearch-0-90-0-returning-incorrect-results/11795/8 "2013-05-10T18:50:48Z")
On 5/8/2013 9:29 AM, Eric Jain wrote: > On Wed, May 8, 2013 at 5:37 AM, Marcin Dojwa [m.dojwa@livechatinc.com](mailto:m.dojwa@livechatinc.com) wrote: > > > Thanks for this one. What happens if you give '2012-11-30T15:00:00Z'? The > > given datetime is treated as UTC or default timezone too ? > > If your local timezone offset is e.g. -07:00, doc['timestamp'].date is > > now (0.90) '2012-11-30T08:00:00-07:00'. Previously (0.20), it would > > have been '2012-11-30T15:00:00Z'; not because elasticsearch was > > honoring the date's original timezone offset (I wish it did!), but > > because the `date` property in a script would be always be UTC. Actually both of those are the same millisecond value stored in the date field displayed differently (In the Summer 8 AM in Seattle is the same moment in time as 3 PM in London) Therefore, it depends on the definition of what the script aDateField.date is supposed to correspond to in the Joda date library and thus what toString method is used to convert the results back to the caller. It is not a matter of honoring the original TZ, because it can't, the field stores a millisecond value; that's it. On the way out nothing knows how it got that millisecond value into the field. As Eric as observed it WAS converting the millisecond value to the result string using UTC. but now it IS converting to the millisecond value to the result string using the DEFAULT VM timezone. "The default date parsing used if no format is specified is ISODateTimeFormat.dateOptionalTimeParser [http://joda-time.sourceforge.net/api-release/org/joda/time/format/ISODateTimeFormat.html#dateOptionalTimeParser()](http://joda-time.sourceforge.net/api-release/org/joda/time/format/ISODateTimeFormat.html#dateOptionalTimeParser%28%29)." "... The first format will also act as the one that converts back from milliseconds to a string representation. ..." -- [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/date-format/) None of the formats specified make any claim as to what TZ will be used when formatting the millisecond value into a readable string, but certainly being able to pick one by setting default VM timezone is a valid choice, but so is the earlier idea of always using UTC, so I don't know which behavior is intended. It is also important that all output formats do the same thing and not mix VM default with UTC. One idea is that forcing UTC as the default on the way out reminds folks to think about what they want to see and to learn the difference between how it is stored and how it is transported back (hopefully it is short leap of logic to thinking about how it is transported into the index). Just as a warning to all, having client and server us different timezones can make hard to find bugs when using the default input format which allows dates (without times) sent from clients generating local dates and then falling into the previous or next day, because they end up at "somebody else's" (the servers) midnight when it is converted for storage into a millisecond value. Client says "2013-05-10" (thinking that means "2012-05-10 T 00:00:00 -07:00" or "2012-05-10 T 07:00:00Z"), server records a timestamp of 7 hours earlier "2012-05-10 T _00_:00:00 _Z_". If later the client code fully specifies things with times and timezones, things can be curiously mismatched with the original timestamp and get subtly confusing. -Paul -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Eric\_Jain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eric_jain/32/834_2.png) [@Eric\_Jain](https://discuss.elastic.co/u/Eric_Jain)\ **Post date:** [May 25, 2013, 12:33am UTC](https://discuss.elastic.co/t/script-terms-facet-in-elasticsearch-0-90-0-returning-incorrect-results/11795/9 "2013-05-25T00:33:34Z")
On Thursday, May 9, 2013 1:45:13 AM UTC-7, Marcin Dojwa wrote: > For me it sounds like a bug. Did you report the issue here > [Issues · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues) ? > > > This may be on purpose, but it's a breaking change from 0.20.6, where dates were… always UTC. See discussion at https://groups.google.com/d/topic/elasticsearch/a\_Jj7d4Uk6c/discussion -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:34am UTC](https://discuss.elastic.co/t/script-terms-facet-in-elasticsearch-0-90-0-returning-incorrect-results/11795/10 "2017-07-06T02:34:51Z")