How to set a socks5 proxy for ElasticsearchTemplate (Java Client)

[blalasaadri (Alasdair Collinson)]

Hi!

I'm trying to connect to a remote Elasticsearch instance; to do so, I have to tunnel my traffic through a socks5 proxy.

In my Java code, I'm using the ElasticsearchTemplate which in turn uses a PreBuiltTransportClient, but I can't find how to set up a proxy for the connection. There seem to be ways to set a proxy when using the RestClient (as described here) but I don't see how that would be used with the ElasticsearchTemplate. Maybe that's the wrong idea anyway. Any hints? Thanks!

[dadoonet (David Pilato)]

I don't have the answer for this TBH.
May be it's just a JAVA_OPTS thingy that you need to set. Like explained for our plugin manager: https://www.elastic.co/guide/en/elasticsearch/plugins/current/_other_command_line_parameters.html#_proxy_settings

JAVA_OPTS="-Dhttp.proxyHost=host_name -Dhttp.proxyPort=port_number -Dhttps.proxyHost=host_name -Dhttps.proxyPort=https_port_number"

But anyway, it's not recommended anymore to use the TransportClient as it will be deprecated then removed in the future.
You should switch to the HighLevel Rest client.

I hope this can help. May be @spinscale has other ideas regarding the proxy settings for Transport Client?

[spinscale (Alexander Reelsen)]

No, we dont support anything in the TransportClient for that.

Even though java has native system properties for socks named socksProxyHostand socksProxyPort I have never tried this and we do not test for this in our code.

On top of trying the above system properties you try could use ssh or another socks proxy (not sure if stunnel can do this) to create a socks tunnel and then point the transport client to that one. I think however that this will result in problems when publish addresses from the cluster state are read and not reachable, so routing is pretty important (and tricky!) here. I am unsure if this will work at all, TBH.

--Alex

[blalasaadri (Alasdair Collinson)]

Thanks for the feedback, we've switched to the HighLevel client. However, that only seems to allow setting an HTTP proxy rather than a SOCKS5 proxy. What I'm trying is the following:

public RestHighLevelClient restClient() {
    RestClientBuilder builder = RestClient.builder(new HttpHost(esHost, esPort, "http"));
    builder.setHttpClientConfigCallback(
            httpAsyncClientBuilder -> httpAsyncClientBuilder.setProxy(new HttpHost(proxyHost, proxyPort))
    );
    return new RestHighLevelClient(builder.build());
}

Every time I then try to access the Elasticsearch instance I get this exception:

org.apache.http.ConnectionClosedException: Connection closed
	at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.endOfInput(HttpAsyncRequestExecutor.java:347) ~[httpcore-nio-4.4.9.jar:4.4.9]
	at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:261) ~[httpcore-nio-4.4.9.jar:4.4.9]
	at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81) ~[httpasyncclient-4.1.3.jar:4.1.3]
	at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39) ~[httpasyncclient-4.1.3.jar:4.1.3]
	at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114) ~[httpcore-nio-4.4.9.jar:4.4.9]
	at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162) ~[httpcore-nio-4.4.9.jar:4.4.9]
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337) ~[httpcore-nio-4.4.9.jar:4.4.9]
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315) ~[httpcore-nio-4.4.9.jar:4.4.9]
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276) ~[httpcore-nio-4.4.9.jar:4.4.9]
	at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) ~[httpcore-nio-4.4.9.jar:4.4.9]
	at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588) ~[httpcore-nio-4.4.9.jar:4.4.9]
	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]

When leaving out the proxy settings (builder.setHttpClientConfigCallback(...)) and connecting to an instance that does not require a proxy, no problem.

We do have a workaround, but getting it to run without that would be great.