Have you set es.nodes.wan.only to true? Using es-hadoop/spark through cloud, elasticsearch appears to be a single node to es-hadoop. You might want to take a look at Writing PySpark dataframe to Elastic Cloud (Cannot detect ES version) for a discussion about some of the complications that come with that.