Ok, because you are trying to sort on nested documents you will need to use the nested_path option. Information on how to use it including an example can be found here: https://www.elastic.co/guide/en/elasticsearch/reference/2.4/search-request-sort.html#nested-sorting
Sorry I didn't mention this earlier, it took me a little too long to understand what you are doing 