This refers to this new feature that began in ES 6.x: https://www.elastic.co/guide/en/elasticsearch/reference/master/index-modules-index-sorting.html
I'm not sure if this is a Lucene issue or Elasticsearch issue, but sorting on a sorted index in the opposite direction should be nearly as fast as sorting on the same direction.
What I'm noticing is that if I set the sort direction to "desc" on a specific field, searches using that field and sort direction are extremely fast, but sorting on that same field with "asc" is much slower.
If Lucene is already holding a sorted index (regardless of sort direction), sorting in the opposite direction should be nearly as fast as sorting in the same sort direction of the actual index.