Several date math questions

[colings86 (Colin Goodheart-Smithe)]

@GordonM

What is the native time zone and format that an Elasticsearch date field is converted into?

All dates are indexed as a long field with a value which is the epoch time in milliseconds, so effectively the dates are converted to UTC.

@GordonM

Is there a way to see a date or timestamp field how Elasticsearch sees it?

If you have enough memory to load the date field into the field data cache then you could use the 'field_data fields' feature to see the indexed value, but I would only advise using that as a debugging option here and not as your final solution.

@GordonM

I am also having problems with getting weekly data, such as last week’s data, because I am not sure how to specify that that Sunday is the beginning of the week over Joda’s default Monday.

I tried this:

"range": {
"timestamp" : {
“gte”: “now/w-1d”,
"lt": "now"
}
}

"gte": "now/w-1d" (read as: round to the beginning of this week (Monday) and subtract one day) may generally work unless it is currently Sunday, because then a week would be Sunday to Sunday instead of Sunday to Saturday.

Would the following work instead?

"range": {
   "timestamp" : {
       “gte”: “now/w-1d”,
        "lt": "(now+1w)/w - 1d"
   }
}

This should find the beginning of the current week for the start of the range and the beginning of the next week for the end of the range.

@GordonM

Is something similar available for simple bool range queries?

As far as I know there isn't an offset for range queries since you have complete control over the start and end of the range. In the date_histogram aggregation the offset parameter is used to adjust where the buckets start from but this doesn't really apply when using a single filter.

@GordonM

Alternatively, is there a setting (global or otherwise) that I can adjust to make the start of a week treated in the standard US locale way?

I don't think there is, sorry.