Java Client and Elasticsearch Transport Proxy

Elastic Stack Elasticsearch
1

Hi,

I have an Elasticsearch cluster running in the cloud and I am trying to
connect from a java client locally. Due to firewall restrictions I am not
allowed to connect out on port 9300 from my local client. I have created a
proxy in the cloud to forward all TCP connections from port 80 to port 9300
on the cluster master node. My client has the same cluster name as the
master.

In my Java code I have

Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name",
"mycluster").build();
TransportClient client = new TransportClient(settings);
client.addTransportAddress(new
InetSocketTransportAddress("{my-proxy-host-ip}", 80));

When trying to access the client I get

[10:00:44] netty - [Hellion] exception caught on transport layer [[id:
0x2882f1d8, /145.36.217.4:49318 => /{my-proxy-host-ip}:80]], closing
connection
java.io.StreamCorruptedException: invalid internal transport message format
at
org.elasticsearch.transport.netty.SizeHeaderFrameDecoder.decode(SizeHeaderFrameDecoder.java:46)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:425)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
at
org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at
org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at
org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)

(I have confirmed the route to the master node via the proxy by changing
the outbound port on the proxy server to 9200 and successfully testing curl
{my-proxy-host-ip} OK)

I can't see any reason why this shouldn't work so any ideas are welcomed.

Thanks in advance.
Kim

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c356c021-9d01-4ed4-926e-85ebaa7bc7a5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

You have configured a HTTP proxy. The transport protocol is binary and not
HTTP.

What you want is a SOCKS proxy.

Jörg

On Wed, Dec 10, 2014 at 11:10 AM, Kim Baddeley kim.baddeley@gmail.com
wrote:

Hi,

I have an Elasticsearch cluster running in the cloud and I am trying to
connect from a java client locally. Due to firewall restrictions I am not
allowed to connect out on port 9300 from my local client. I have created a
proxy in the cloud to forward all TCP connections from port 80 to port 9300
on the cluster master node. My client has the same cluster name as the
master.

In my Java code I have

Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name",
"mycluster").build();
TransportClient client = new TransportClient(settings);
client.addTransportAddress(new
InetSocketTransportAddress("{my-proxy-host-ip}", 80));

When trying to access the client I get

[10:00:44] netty - [Hellion] exception caught on transport layer [[id:
0x2882f1d8, /145.36.217.4:49318 => /{my-proxy-host-ip}:80]], closing
connection
java.io.StreamCorruptedException: invalid internal transport message format
at
org.elasticsearch.transport.netty.SizeHeaderFrameDecoder.decode(SizeHeaderFrameDecoder.java:46)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:425)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
at
org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at
org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at
org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)

(I have confirmed the route to the master node via the proxy by changing
the outbound port on the proxy server to 9200 and successfully testing curl
{my-proxy-host-ip} OK)

I can't see any reason why this shouldn't work so any ideas are welcomed.

Thanks in advance.
Kim

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c356c021-9d01-4ed4-926e-85ebaa7bc7a5%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/c356c021-9d01-4ed4-926e-85ebaa7bc7a5%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFHxDCqX2m%3DRXwNbLwmOYPobASSZ6iJrfPipnE8VaEz2g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

I have configured a TCP proxy not an HTTP proxy. Have tried this using both
AWS ELB and EC2 running "socat LISTEN-TCP:80,fork TCP:10.178.241.66:9300"
with the same result.

Kim

On Wednesday, 10 December 2014 13:46:14 UTC, Jörg Prante wrote:

You have configured a HTTP proxy. The transport protocol is binary and not
HTTP.

What you want is a SOCKS proxy.

Jörg

On Wed, Dec 10, 2014 at 11:10 AM, Kim Baddeley <kim.ba...@gmail.com
<javascript:>> wrote:

Hi,

I have an Elasticsearch cluster running in the cloud and I am trying to
connect from a java client locally. Due to firewall restrictions I am not
allowed to connect out on port 9300 from my local client. I have created a
proxy in the cloud to forward all TCP connections from port 80 to port 9300
on the cluster master node. My client has the same cluster name as the
master.

In my Java code I have

Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name",
"mycluster").build();
TransportClient client = new TransportClient(settings);
client.addTransportAddress(new
InetSocketTransportAddress("{my-proxy-host-ip}", 80));

When trying to access the client I get

[10:00:44] netty - [Hellion] exception caught on transport layer [[id:
0x2882f1d8, /145.36.217.4:49318 => /{my-proxy-host-ip}:80]], closing
connection
java.io.StreamCorruptedException: invalid internal transport message
format
at
org.elasticsearch.transport.netty.SizeHeaderFrameDecoder.decode(SizeHeaderFrameDecoder.java:46)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:425)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
at
org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at
org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at
org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)

(I have confirmed the route to the master node via the proxy by changing
the outbound port on the proxy server to 9200 and successfully testing curl
{my-proxy-host-ip} OK)

I can't see any reason why this shouldn't work so any ideas are welcomed.

Thanks in advance.
Kim

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c356c021-9d01-4ed4-926e-85ebaa7bc7a5%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/c356c021-9d01-4ed4-926e-85ebaa7bc7a5%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

On Wednesday, 10 December 2014 13:46:14 UTC, Jörg Prante wrote:

You have configured a HTTP proxy. The transport protocol is binary and not
HTTP.

What you want is a SOCKS proxy.

Jörg

On Wed, Dec 10, 2014 at 11:10 AM, Kim Baddeley <kim.ba...@gmail.com
<javascript:>> wrote:

Hi,

I have an Elasticsearch cluster running in the cloud and I am trying to
connect from a java client locally. Due to firewall restrictions I am not
allowed to connect out on port 9300 from my local client. I have created a
proxy in the cloud to forward all TCP connections from port 80 to port 9300
on the cluster master node. My client has the same cluster name as the
master.

In my Java code I have

Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name",
"mycluster").build();
TransportClient client = new TransportClient(settings);
client.addTransportAddress(new
InetSocketTransportAddress("{my-proxy-host-ip}", 80));

When trying to access the client I get

[10:00:44] netty - [Hellion] exception caught on transport layer [[id:
0x2882f1d8, /145.36.217.4:49318 => /{my-proxy-host-ip}:80]], closing
connection
java.io.StreamCorruptedException: invalid internal transport message
format
at
org.elasticsearch.transport.netty.SizeHeaderFrameDecoder.decode(SizeHeaderFrameDecoder.java:46)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:425)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
at
org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at
org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at
org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)

(I have confirmed the route to the master node via the proxy by changing
the outbound port on the proxy server to 9200 and successfully testing curl
{my-proxy-host-ip} OK)

I can't see any reason why this shouldn't work so any ideas are welcomed.

Thanks in advance.
Kim

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c356c021-9d01-4ed4-926e-85ebaa7bc7a5%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/c356c021-9d01-4ed4-926e-85ebaa7bc7a5%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/053a48bc-49d2-4cf2-b261-ad399bf8de64%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.