Conditional sorting on multi-field value

I've a object which has a name, to be able to sort on it I'm using keyword-lowercase analyzer.
The sorting on this object itself works perfectly fine using the full path like name.raw.
My problem now is that the object is a nested object and there's a second, equal object (structurally, not content) as well (Object A contains nested Object B twice).
Only if the searching user is the author of the first he can find it.
All the filtering to select the right objects works perfectly fine, when sorting though I needed to use a script as by simply applying filters to the sorts it just doesn't include non matching documents but still applies each sort operation on every document instead of sorting on the superset.
My sorting script is quite simple and works ("type": "string"):
if (_source.b1 && _source.authorId==authorId) {_source.b1.name} else {_source.b2.name}

If the searching user is the author I want to use b1.name for sorting and otherwise b2.name.
The problem with this now is that it really should be b1.name.raw and b2.name.raw but that throws an exception saying no property raw.
My mapping is correct I assume (as it's the same as before and it worked):

"name": {
"type": "string",
"fields": {
"name.raw": {
"type": "string",
"index_analyzer": "lowercase_sort"
}
}
}

My current workaround is to either call toLowerCase() on the names in the script, or (my preferred) have another non-multi-field value which is simply not analyzed but lowercased when putting it in the index.

I looked at all kinds of variations on path, tried to use doc, _index but can't get to the lowercased string.

Thanks