My bad, you're right. Tim wrote about this in the past here: Kibana Filter for a specific time range
You would have to extract that into something that is filterable, like a boolean field "is_working_hours". You could use a scripted field, but that wouldn't be very performant.