In this particular case, the issue is an unintentional side effect of refactoring. However, it does fall in line with long suggested guidance, which is accessing _source in scoring scripts is slow and should not be done.
Do you have doc values disabled for categories and category_id?