Well, it turned out to be a stupid error on my part. I had two instances of ES running apparently on the same machine and restarting the service had no effect since there was another instance blocking the reloads...
It seems ES is quietly just finding the next unoccupied port number and will not even give a warning that the standard port is blocked.