Insert into elastic from spark: Connection error - all nodes failed

Elastic Stack Elasticsearch
1

Hi,
I tried to insert 580 docs(~2MB size) to es but I got this message,
when i tried to decrease the number of docs to 50 the job success.
I am working with one spark partition, tried to tweak the batch size and the batch entries,
but didn't succeed to overcome this...
I also succeeded to connect to the es nodes from spark.

org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed;[nodes with ip and port]

Some details:
4 nodes of spark and elastic (4 cores 24GB, elastic heap size 2GB)
spark 1.6.0
es 2.3.3
es-hadoop 2.4.2

Thanks.

2

@Netanel_Malka In 5.0 and above, the connector logs a sample of errors that it receives from the Elasticsearch cluster. In the mean time, I would increase logging levels to TRACE for your job and inspecting the error messages on the bulk responses. Also be sure to check your Elasticsearch node logs for rejections.

3

Thanks for the respond.
I found that only when I try to insert a doc with big(thousands of coordinates) geo_shape(), I got this error.
Also, i tried it in elasticsearch 5.2.2 with the suite connector, but the connector not write why it failed.
I see that it finds the appropriate shards(DEBUG) and the partition writer instance on the right address, but then i got this exception again.
Also, When i tried to delete the index i get acknowledge: false, although the index is deleted. But then i can't create any index until I restart the whole cluster...

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.