To quick-fix the issue, I tried just adding a bucket script aggregation, which turns out, suffers the same issue I am seeing here I think.
I actually think the issue may be higher up, in Elasticsearch: Bucket Script Aggregating on fields that contain a period . To narrow this down, I made a small example query. This may not be VEGA-related after all, but just ES related.