Pre_zone_adjust_large_interval is not working

Marcin_Dojwa · March 30, 2012, 1:54pm

Hi,

I try to use "pre_zone_adjust_large_interval":true with 'date_histogram' as
follows:
{
"size":0,
"facets":{
"rated_good":{
"date_histogram":{
"pre_zone_adjust_large_interval":true,
"field":"chat.start_date",
"interval":"day",
"pre_zone":"Europe/Warsaw"
},
"facet_filter":{
"and":[
{
"range":{
"chat.start_date":{
"from":"2012-03-28T23:46:26Z",
"to":"2012-03-28T23:46:27Z"
}
}
},
{
"term":{
"chat.rate":2
}
}
]
}
}
}
}

This request will find one document that has chat.start_date =
"2012-03-28T23:46:26Z" (in Europe/Warsaw it is "2012-03-29T01:46:26"). The
datetime in the response should be: "2012-03-28T22:00:00Z" right? (it
is 2012-03-29T00:00:00Z converted to 2012-03-28T22:00:00Z due
to pre_zone_adjust_large_interval set to true and pre_zone
"Europe/Warsaw"), but as you can see below it is 1332979200000 which is
exactly 2012-03-29T00:00:00Z as it is when not
using pre_zone_adjust_large_interval. Am I doing something wrong (most
probably :))?

The result is of the request above is:
{
"took" : 11,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 1048608,
"max_score" : 1.0,
"hits" : [ ]
},
"facets" : {
"rated_good" : {
"_type" : "date_histogram",
"entries" : [ {
"time" : 1332979200000,
"count" : 1
} ]
}
}
}

Thanks for help.
Best regards
Marcin.

Marcin_Dojwa · March 30, 2012, 1:55pm

I forgot to add that I am using ES 0.19.1 (it behaved the same on ES
0.19.0.RC3)

2012/3/30 Marcin Dojwa m.dojwa@livechatinc.com

Hi,

I try to use "pre_zone_adjust_large_interval":true with 'date_histogram'
as follows:
{
"size":0,
"facets":{
"rated_good":{
"date_histogram":{
"pre_zone_adjust_large_interval":true,
"field":"chat.start_date",
"interval":"day",
"pre_zone":"Europe/Warsaw"
},
"facet_filter":{
"and":[
{
"range":{
"chat.start_date":{
"from":"2012-03-28T23:46:26Z",
"to":"2012-03-28T23:46:27Z"
}
}
},
{
"term":{
"chat.rate":2
}
}
]
}
}
}
}

This request will find one document that has chat.start_date =
"2012-03-28T23:46:26Z" (in Europe/Warsaw it is "2012-03-29T01:46:26"). The
datetime in the response should be: "2012-03-28T22:00:00Z" right? (it
is 2012-03-29T00:00:00Z converted to 2012-03-28T22:00:00Z due
to pre_zone_adjust_large_interval set to true and pre_zone
"Europe/Warsaw"), but as you can see below it is 1332979200000 which is
exactly 2012-03-29T00:00:00Z as it is when not
using pre_zone_adjust_large_interval. Am I doing something wrong (most
probably :))?

The result is of the request above is:
{
"took" : 11,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 1048608,
"max_score" : 1.0,
"hits" :
},
"facets" : {
"rated_good" : {
"_type" : "date_histogram",
"entries" : [ {
"time" : 1332979200000,
"count" : 1
} ]
}
}
}

Thanks for help.
Best regards
Marcin.

Marcin_Dojwa · March 30, 2012, 5:21pm

Maybe someone could show me working example with pre_zone_adjust_large_interval
? I could try to work this out by myself then.

Thanks.

W dniu piątek, 30 marca 2012 użytkownik Marcin Dojwa napisał:

I forgot to add that I am using ES 0.19.1 (it behaved the same on ES
0.19.0.RC3)

2012/3/30 Marcin Dojwa <m.dojwa@livechatinc.com <javascript:_e({},
'cvml', 'm.dojwa@livechatinc.com');>>

Hi,

I try to use "pre_zone_adjust_large_interval":true with 'date_histogram'
as follows:
{
"size":0,
"facets":{
"rated_good":{
"date_histogram":{
"pre_zone_adjust_large_interval":true,
"field":"chat.start_date",
"interval":"day",
"pre_zone":"Europe/Warsaw"
},
"facet_filter":{
"and":[
{
"range":{
"chat.start_date":{
"from":"2012-03-28T23:46:26Z",
"to":"2012-03-28T23:46:27Z"
}
}
},
{
"term":{
"chat.rate":2
}
}
]
}
}
}
}

This request will find one document that has chat.start_date =
"2012-03-28T23:46:26Z" (in Europe/Warsaw it is "2012-03-29T01:46:26"). The
datetime in the response should be: "2012-03-28T22:00:00Z" right? (it
is 2012-03-29T00:00:00Z converted to 2012-03-28T22:00:00Z due
to pre_zone_adjust_large_interval set to true and pre_zone
"Europe/Warsaw"), but as you can see below it is 1332979200000 which is
exactly 2012-03-29T00:00:00Z as it is when not
using pre_zone_adjust_large_interval. Am I doing something wrong (most
probably :))?

The result is of the request above is:
{
"took" : 11,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 1048608,
"max_score" : 1.0,
"hits" :
},
"facets" : {
"rated_good" : {
"_type" : "date_histogram",
"entries" : [ {
"time" : 1332979200000,
"count" : 1
} ]
}
}
}

Thanks for help.
Best regards
Marcin.

Eric_Jain · March 30, 2012, 6:03pm

On Mar 30, 10:21 am, Marcin Dojwa m.do...@livechatinc.com wrote:

Maybe someone could show me working example with pre_zone_adjust_large_interval
? I could try to work this out by myself then.

I'm also using 0.19.1 and do:

"facets" : {
"timeline" : {
"date_histogram" : {
"field" : "timestamp",
"interval" : "day",
"pre_zone" : "US/Pacific",
"pre_zone_adjust_large_interval" : true
}
}
}

This returns e.g.

"facets" : {
"timeline" : {
"_type" : "date_histogram",
"entries" : [ {
"time" : 1301554800000,
"count" : 1
} ]
}
}

1301554800000 == 2011-03-31T00:00:00.000-07:00

I don't have a facet_filter, maybe that is causing the problem?

Marcin_Dojwa · March 31, 2012, 11:54am

Hi, thanks for response. Unfortunately this is not a problem of facet
filters. The following gives the same problem:

{
"size":0,
"facets":{
"rated_good":{
"date_histogram":{
"field":"chat.start_date",
"interval":"day",
"time_zone":"Europe/Warsaw",
"pre_zone_adjust_large_interval":true
}
}
}
}

A part of response:
{
"time" : 1332892800000,
"count" : 360
}

1332892800000 -> 2012-03-28T00:00:00Z

So the response is not right. Maybe there is anything in config file that I
should change ? (I use the default one).

Best regards.

2012/3/30 Eric Jain eric.jain@gmail.com

On Mar 30, 10:21 am, Marcin Dojwa m.do...@livechatinc.com wrote:

Maybe someone could show me working example with
pre_zone_adjust_large_interval
? I could try to work this out by myself then.

I'm also using 0.19.1 and do:

"facets" : {
"timeline" : {
"date_histogram" : {
"field" : "timestamp",
"interval" : "day",
"pre_zone" : "US/Pacific",
"pre_zone_adjust_large_interval" : true
}
}
}

This returns e.g.

"facets" : {
"timeline" : {
"_type" : "date_histogram",
"entries" : [ {
"time" : 1301554800000,
"count" : 1
} ]
}
}

1301554800000 == 2011-03-31T00:00:00.000-07:00

I don't have a facet_filter, maybe that is causing the problem?

Eric_Jain · March 31, 2012, 5:42pm

On Sat, Mar 31, 2012 at 04:54, Marcin Dojwa m.dojwa@livechatinc.com wrote:

So the response is not right. Maybe there is anything in config file that I
should change ? (I use the default one).

I don't have any special configuration, either.

Here's the unit test I use to verify that preZoneAdjustLargeInterval
works the way I expect it to:

gist.github.com

https://gist.github.com/ejain/2267024

DateHistogramFacetTest.java

package search;

import java.util.Map;

import junit.framework.Assert;

import org.elasticsearch.action.search.SearchRequestBuilder;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.Client;
import org.elasticsearch.common.settings.ImmutableSettings;

This file has been truncated. show original

Marcin_Dojwa · March 31, 2012, 6:59pm

Hi, thanks, but I can not run this unit test. To be sure that I did not
missed anything I did a simple example:

curl -XPUT 'http://localhost:9200/twitter/tweet/1' -d '
{
"user":"kimchy",
"post_date":"2009-11-15T14:12:12Z",
"message":"trying out Elastic Search"
}
'

curl -X POST "http://localhost:9200/twitter/tweet/_search?pretty=true" -d '
{
"query":{
"match_all":{

},
"facets":{
"histo1":{
"date_histogram":{
"field":"post_date",
"interval":"day",
"pre_zone":"Europe/Warsaw",
"pre_zone_adjust_large_interval":true
}
}
}
}
'

The result of 2:
{
"took":2,
"timed_out":false,
"_shards":{
"total":5,
"successful":5,
"failed":0
},
"hits":{
"total":1,
"max_score":1.0,
"hits":[
{
"_index":"twitter",
"_type":"tweet",
"_id":"1",
"_score":1.0,
"_source":{
"user":"kimchy",
"post_date":"2009-11-15T14:12:12",
"message":"trying out Elastic Search"
}
}
]
},
"facets":{
"histo1":{
"_type":"date_histogram",
"entries":[
{
"time":1258243200000,
"count":1
}
]
}
}
}

Time: 1258243200000 > 2009-11-15T00:00:00 UTC but it should be
2009-11-14T22:00:00
UTC (Europe/Warsaw is GMT+2 currently - CEST)

I have no idea what could be wrong Does anyone know ?

Thanks for help.
Best regards.
Marcin.

2012/3/31 Eric Jain eric.jain@gmail.com

On Sat, Mar 31, 2012 at 04:54, Marcin Dojwa m.dojwa@livechatinc.com
wrote:

So the response is not right. Maybe there is anything in config file
that I
should change ? (I use the default one).

I don't have any special configuration, either.

Here's the unit test I use to verify that preZoneAdjustLargeInterval
works the way I expect it to:

Unit test for using elasticsearch's DateHistogramFacet with time zones. · GitHub

Marcin_Dojwa · April 2, 2012, 2:32pm

Unfortunately it looks like it's not working at all here I did a
workaround at my end.

Best regards.

2012/3/31 Marcin Dojwa m.dojwa@livechatinc.com

Hi, thanks, but I can not run this unit test. To be sure that I did not
missed anything I did a simple example:

curl -XPUT 'http://localhost:9200/twitter/tweet/1' -d '
{
"user":"kimchy",
"post_date":"2009-11-15T14:12:12Z",
"message":"trying out Elastic Search"
}
'

curl -X POST "http://localhost:9200/twitter/tweet/_search?pretty=true" -d
'
{
"query":{
"match_all":{
  }
},
"facets":{
"histo1":{
"date_histogram":{
"field":"post_date",
"interval":"day",
"pre_zone":"Europe/Warsaw",
"pre_zone_adjust_large_interval":true
}
}
}
}
'

The result of 2:
{
"took":2,
"timed_out":false,
"_shards":{
"total":5,
"successful":5,
"failed":0
},
"hits":{
"total":1,
"max_score":1.0,
"hits":[
{
"_index":"twitter",
"_type":"tweet",
"_id":"1",
"_score":1.0,
"_source":{
"user":"kimchy",
"post_date":"2009-11-15T14:12:12",
"message":"trying out Elastic Search"
}
}
]
},
"facets":{
"histo1":{
"_type":"date_histogram",
"entries":[
{
"time":1258243200000,
"count":1
}
]
}
}
}

Time: 1258243200000 > 2009-11-15T00:00:00 UTC but it should be 2009-11-14T22:00:00
UTC (Europe/Warsaw is GMT+2 currently - CEST)

I have no idea what could be wrong Does anyone know ?

Thanks for help.
Best regards.
Marcin.

2012/3/31 Eric Jain eric.jain@gmail.com

On Sat, Mar 31, 2012 at 04:54, Marcin Dojwa m.dojwa@livechatinc.com
wrote:

So the response is not right. Maybe there is anything in config file
that I
should change ? (I use the default one).

I don't have any special configuration, either.

Here's the unit test I use to verify that preZoneAdjustLargeInterval
works the way I expect it to:

Unit test for using elasticsearch's DateHistogramFacet with time zones. · GitHub

Eric_Jain · April 2, 2012, 5:18pm

On Sat, Mar 31, 2012 at 11:59, Marcin Dojwa m.dojwa@livechatinc.com wrote:

Time: 1258243200000 > 2009-11-15T00:00:00 UTC but it should
be 2009-11-14T22:00:00 UTC (Europe/Warsaw is GMT+2 currently - CEST)

I just ran the two curl requests on a fresh install of elasticsearch
0.19.1, and the facet time that is returned is 1258239600000. So there
must be another problem?