@pavocristatus Please avoid cross posting from StackOverflow. This makes responding to issues difficult when information is split across two sites. Additionally, readers that may be experiencing your problem may have a hard time finding this post if it is not the same content. Going forward, I request that you post the issue/question in full on here.
I suggest to avoid using the Transport client with Spark. The Transport client depends on Netty, which Spark also depends on. We ran into issues at the start of the 5.0 line with our testing in ES-Hadoop because the Netty versions in Spark and Elasticsearch did not agree with each other. I would start by suggesting to use the Rest Client that is now provided in Java to see if that helps any.