Hour Scripted Field

[stevezemlicka (Stephen Zemlicka)]

I am trying to create a scripted field that is the hourly value of each document. The ultimate goal is to filter by specific hours of the day for multiple days (hence why I can't just use the date range on the dashboards).

After a bit of research, the best resource I found was the following which seems to indicate that this is possible:

Scripted field HourOfDay()

Hi, I want a field containing the hour of the day for each elasticsearch document. The easiest way would be to parse @timestamp into a hour_of_day field but that means I would have to reindex all my data. I saw another solution by using Scripted Fields with: doc["@timestamp"].getHourOfDay() That works to get the hour ('HH') in format 00-23 and that's exactly what I was expecting. But, the hour_of_day field isn't correct as it shows a different hour value (probably due to timezone difference…

However I cannot seem to get it to work. When I try to preview the results, I receive the following:

doc['@timestamp'].date.hourOfDay

produces

{
"root_cause": [
{
"type": "script_exception",
"reason": "runtime error",
"script_stack": [
"doc['@timestamp'].date.hourOfDay",
" ^---- HERE"
],
"script": "doc['@timestamp'].date.hourOfDay",
"lang": "painless",
"position": {
"offset": 17,
"start": 0,
"end": 32
}
}
],
"type": "search_phase_execution_exception",
"reason": "all shards failed",
"phase": "query",
"grouped": true,
"failed_shards": [
{
"shard": 0,
"index": "san-isabel",
"node": "erbPD7dEQPKwU5NNOXVB9g",
"reason": {
"type": "script_exception",
"reason": "runtime error",
"script_stack": [
"doc['@timestamp'].date.hourOfDay",
" ^---- HERE"
],
"script": "doc['@timestamp'].date.hourOfDay",
"lang": "painless",
"position": {
"offset": 17,
"start": 0,
"end": 32
},
"caused_by": {
"type": "illegal_argument_exception",
"reason": "Illegal list shortcut value [date]."
}
}
}
]
}

and this:

LocalDateTime.ofInstant(Instant.ofEpochMilli(doc['@timestamp'].value),ZoneId.of('America/Chicago')).getHour()

produces

{
"root_cause": [
{
"type": "script_exception",
"reason": "runtime error",
"script_stack": [
"LocalDateTime.ofInstant(Instant.ofEpochMilli(doc['@timestamp'].value), ZoneId.of('America/Chicago')).getHour()",
" ^---- HERE"
],
"script": "LocalDateTime.ofInstant(Instant.ofEpochMilli(doc['@timestamp'].value), ZoneId.of('America/Chicago')).getHour()",
"lang": "painless",
"position": {
"offset": 62,
"start": 0,
"end": 110
}
}
],
"type": "search_phase_execution_exception",
"reason": "all shards failed",
"phase": "query",
"grouped": true,
"failed_shards": [
{
"shard": 0,
"index": "san-isabel",
"node": "erbPD7dEQPKwU5NNOXVB9g",
"reason": {
"type": "script_exception",
"reason": "runtime error",
"script_stack": [
"LocalDateTime.ofInstant(Instant.ofEpochMilli(doc['@timestamp'].value), ZoneId.of('America/Chicago')).getHour()",
" ^---- HERE"
],
"script": "LocalDateTime.ofInstant(Instant.ofEpochMilli(doc['@timestamp'].value), ZoneId.of('America/Chicago')).getHour()",
"lang": "painless",
"position": {
"offset": 62,
"start": 0,
"end": 110
},
"caused_by": {
"type": "wrong_method_type_exception",
"reason": "cannot convert MethodHandle(Dates)JodaCompatibleZonedDateTime to (Object)long"
}
}
}
]
}

What might I be doing wrong and how might I accomplish this?

[stevezemlicka (Stephen Zemlicka)]

It might be helpful to provide a bit more context on my data. When ingesting, I am mapping a "date" field to @timestamp. So while I do have @timestamp, I also have a simpler date field in each document in the following format:

MM/DD/YYYY HH:mm

Perhaps I can pull it from that consistently formated field without relying on any fancy get functions?

[aaron-nimocks]

return doc['@timestamp'].value.getHour();

This works for me.

[stevezemlicka (Stephen Zemlicka)]

@aaron-nimocks

return doc['@timestamp'].value.getHour();

YES!!! This worked for me as well. Thank you!

[stevezemlicka (Stephen Zemlicka)]

For those that come across this in the future, I had an issue with the previous solution and timezone. Seems this was matching UTC which didn't align with the expected output due to how timezones are handled with scripted fields. What ultimately worked was:

return LocalDateTime.ofInstant(Instant.ofEpochMilli(doc['@timestamp'].value.millis),ZoneId.of('America/Chicago')).getHour()

ES 6.0 - Scripted Field HourOfDay with Timezone - error

Hello, First Post Here, I will try to do my best to make a clear presentation. I recently upgraded ELK on my UAT node from 5.6 to 6.0 OS : 2012r2 ; single node. ELK : 6.0.0 Java : 1.8.0_151 I was running a painless scripted field pretty well on 5.6.0 to deduce HourOfDay, as follow : Name : HourOfDay Script : LocalDateTime.ofInstant(Instant.ofEpochMilli(doc['@timestamp'].value),ZoneId.of('Europe/Paris')).getHour() I used this script because of the Timezone I'm using : Europe/Paris ; it…