AWS Elastiseach

I am having a problem with AWS Elasticsearch 5.1.1.

Firstly, I setup manual cluster with 2 nodes with Elasticsearch 5.1.1 on local machines and I used Elasticseach client 5.1.1 and Spark connector and everything was working fine. But when I switched to AWS Elasticsearch 5.1.1 and tried the same thing, it starts giving me errors like Cluster not accessible. e.t.c I found that it's because AWS Elasticsearch does not support transport port. So, I am limited to use only HTTP port 80. When I try rest API it works fine but it's slow because I am ingesting massive data.

Secondly, I tried to ingest directly from Spark using DataFrames it again gave me the error.

Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only'

FYI elastic #cloud supports transport protocol in case it helps.

Moving in #elasticsearch-and-hadoop forum might be better may be to get help about this?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.