Yes the time has decreased, but i forgot to mention (and illustrate in the query i gave ) that the query does differ a bit for highlighting.
Some words like "the", "is", "a", "to" get removed from the highlighting and is a requirement in our case.
But thanks for the feedback.
I found something that might help in this thread.
They mentioned that a fast vector highlighter will decrease the search speeds by 2x at the cost of more disk space being used (not an issue in my case). Sounds promising 