unless I got my test wrong, the overhead of the inner hits wasn't that great and so it was worth getting more URLs per bucket.
So the first result is the for the query without inner_hits ? :
"collapse": {
"field": "hostname"
}
That's not what I'd expect I agree. Do you have consistent results in your benchmark ?
reason": "java.util.concurrent.ExecutionException: CircuitBreakingException[[fielddata] Data too large, data for [_id] would be [1600542593/1.4gb], which is larger than the limit of [622775500/593.9mb]]"
That should not be related to field collapsing. This circuit breaker exception is about loading the field data for the _id field so it is generated by another request.