Spark + Kerberos + SearchGuard + ElasticSearch 1.7.3

Elastic Stack Elasticsearch
1

I am trying to get some Spark Job working runing on ES configured with SearchGuarg ( SPNEGO/Kerberos) . I Dont figure out how to pass Security/Kerberos in Spark Conf

Is there any one who did the test
Thanks

2

If you take ES-Spark out of the equation, how would your Spark configuration look like?

3

("spark.serializer", sparkSerialzer),
("spark.streaming.blockInterval", blockInterval),
("spark.streaming.receiver.maxRate", receiverMaxrate),
("spark.streaming.receiver.write.ahead.log.enable", isWal),
//("es.nodes", nodes),
//("es.port", "9200"),
//("es.batch.size.entries", esBulkSize),
//("es.batch.size.bytes", esBulkByteSize),
//(ConfigurationOptions.ES_NET_HTTP_AUTH_USER, eslogin),
//(ConfigurationOptions.ES_NET_HTTP_AUTH_PASS, esPassword))

4

I meant how do you configure Kerberos/SearchGuard in Spark without ES-Spark? Since ES-Spark is a library, it simply wraps the existing environment and its configuration.

As this is a Spark specific question, I think you'll get more support over at the Spark forums/mailing list.