Es spark with local elasticsearch listening on non default ports

Elastic Stack Elasticsearch
1

Hi,

I am using es-spark_2.11-2.1.0.Beta4 on Spark 1.3.1 shell against
elasticsearch 1.4.4.

I have elasticsearch settings:

http.port: 9201

transport.tcp.port: 9301

I am setting the port in conf/spark-defaults.conf:

spark.es.port 9201

creating an esRdd works fine, however, trying an operation on it, like
esRdd.count, get stuck on recurring timeouts.

Thanks,

Israel

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/41251e78-c2d3-4dc1-a5b2-0d432ea0cdac%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Since the RDD is created lazily, there's no validation done during its
declaration - the actual logic behind it is executed when an operation
(like count) is performed. It looks like the nodes are not properly
accessed; potentially the port is not being picked up.
Can you try passing the es.node configuration through the SparkConf
object and see whether that makes a difference? If that doesn't work,
please enable logging on the rest package [1] and check whether the
proper nodes (and ports) are properly accessed.
If that's not the case, please raise an issue [2]

Cheers,

[1] Logging | Elasticsearch for Apache Hadoop [master] | Elastic
[2] Troubleshooting | Elasticsearch for Apache Hadoop [master] | Elastic

On Thu, Apr 30, 2015 at 6:34 PM, Israel Klein israel.klein@gmail.com wrote:

Hi,

I am using es-spark_2.11-2.1.0.Beta4 on Spark 1.3.1 shell against
elasticsearch 1.4.4.

I have elasticsearch settings:

http.port: 9201

transport.tcp.port: 9301

I am setting the port in conf/spark-defaults.conf:

spark.es.port 9201

creating an esRdd works fine, however, trying an operation on it, like
esRdd.count, get stuck on recurring timeouts.

Thanks,

Israel

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/41251e78-c2d3-4dc1-a5b2-0d432ea0cdac%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAJogdmeNrGmuXx7BAOBpLAC0rFVZND81jy5m6Ez87Yab%2BJLOoQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.