That exception looks like java.io.StreamCorruptedException: invalid internal transport message format, got (16,3,3,0). I don't know if it is relevant to your spark problem or not, but it seems worth fixing. It would also be good to upgrade to a supported version if at all possible.