# Invalid internal transport message format (ff,f4,ff,fd)
**URL:** https://discuss.elastic.co/t/invalid-internal-transport-message-format-ff-f4-ff-fd/34586
**Category:** Elasticsearch
**Created:** [November 14, 2015, 10:51am UTC](https://discuss.elastic.co/t/invalid-internal-transport-message-format-ff-f4-ff-fd/34586 "2015-11-14T10:51:59Z")
**Posts on this page:** 15
**Page:** 1
### Author:  [@vladmiller](https://discuss.elastic.co/u/vladmiller)
#### Post date: [November 14, 2015, 10:51am UTC](https://discuss.elastic.co/t/invalid-internal-transport-message-format-ff-f4-ff-fd/34586/1 "2015-11-14T10:51:59Z")
Hello, so I'm trying to configure ES on AWS EB service. I have two nodes and one of the nodes fails with the following message
> {"error":{"root\_cause":[{"type":"master\_not\_discovered\_exception","reason":"waited for [30s]"}],"type":"master\_not\_discovered\_exception","reason":"waited for [30s]"},"status":503}
When I looked at the error logs I saw
> [2015-11-14 09:23:39,501][WARN][transport.netty] [718e01a158f7] exception caught on transport layer [[id: 0x801cfa11, /10.170.122.175:44259 :\> /172.17.0.5:9300]], closing connection
> java.io.StreamCorruptedException: invalid internal transport message format, got (ff,f4,ff,fd)
> at org.elasticsearch.transport.netty.SizeHeaderFrameDecoder.decode(SizeHeaderFrameDecoder.java:64)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:425)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.cleanup(FrameDecoder.java:482)
> .......
From the code in [SizeHeaderFrameDecoder.java](https://github.com/elastic/elasticsearch/blob/148265bd164cd5a614cd020fb480d5974f523d81/core/src/main/java/org/elasticsearch/transport/netty/SizeHeaderFrameDecoder.java#L49-L69) you can see that ES expects first two bytes to be equal to **ES** , however from tcpdump I can see something different
```
10.151.143.59.39581 > 10.170.122.175.9300: Flags [P.], cksum 0x1ff5 (incorrect -> 0xefbc), seq 1:164, ack 1, win 229, options [nop,nop,TS val 1296347 ecr 1292999], length 163
0x0000: 4500 00d7 d46e 4000 3f06 4787 0a97 8f3b E....n@.?.G....;
0x0010: 0aaa 7aaf 9a9d 2454 4f01 13b7 7d46 d95c ..z...$TO...}F.\
0x0020: 8018 00e5 1ff5 0000 0101 080a 0013 c7db ................
0x0030: 0013 bac7 4553 0000 009d 0000 0000 0000 ....ES..........
0x0040: 0183 0000 1e84 e31e 696e 7465 726e 616c ........internal
0x0050: 3a64 6973 636f 7665 7279 2f7a 656e 2f75 :discovery/zen/u
0x0060: 6e69 6361 7374 0000 0000 4100 0000 00b2 nicast....A.....
0x0070: d05e 000a 6573 2d73 7461 6769 6e67 0c30 .^..es-staging.0
0x0080: 6534 6535 6461 6461 3463 6216 5436 4f31 e4e5dada4cb.T6O1
0x0090: 6368 5932 5233 6938 3452 665f 6178 4236 chY2R3i84Rf_axB6
0x00a0: 4f51 0d32 3535 2e32 3535 2e32 3535 2e30 OQ.255.255.255.0
0x00b0: 0d32 3535 2e32 3535 2e32 3535 2e30 0001 .255.255.255.0..
0x00c0: 04ff ffff 0000 0024 5400 e389 7a00 0000 .......$T...z...
```
As you can see first two bytes are 0x4500 **E.** and not **ES**
Both nodes started from Docker container, have equal environment and cluster name is **es-staging**
How do I fix this problem?
Thanks
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [November 14, 2015, 11:04am UTC](https://discuss.elastic.co/t/invalid-internal-transport-message-format-ff-f4-ff-fd/34586/2 "2015-11-14T11:04:38Z")
Are you running the same version on both nodes? Could you check that?
---
### Author:  [@vladmiller](https://discuss.elastic.co/u/vladmiller)
#### Post date: [November 14, 2015, 4:25pm UTC](https://discuss.elastic.co/t/invalid-internal-transport-message-format-ff-f4-ff-fd/34586/3 "2015-11-14T16:25:02Z")
**Node 1**
> elasticsearch@bb0b992ff048:/$ java -version
> openjdk version "1.8.0\_66-internal"
> OpenJDK Runtime Environment (build 1.8.0\_66-internal-b17)
> OpenJDK 64-Bit Server VM (build 25.66-b17, mixed mode)
```
{
"name" : "bb0b992ff048",
"cluster_name" : "es-staging",
"version" : {
"number" : "2.0.0",
"build_hash" : "de54438d6af8f9340d50c5c786151783ce7d6be5",
"build_timestamp" : "2015-10-22T08:09:48Z",
"build_snapshot" : false,
"lucene_version" : "5.2.1"
},
"tagline" : "You Know, for Search"
}
```
**Node 2**
> openjdk version "1.8.0\_66-internal"
> OpenJDK Runtime Environment (build 1.8.0\_66-internal-b17)
> OpenJDK 64-Bit Server VM (build 25.66-b17, mixed mode)
```
{
"name" : "3b414043c7df",
"cluster_name" : "es-staging",
"version" : {
"number" : "2.0.0",
"build_hash" : "de54438d6af8f9340d50c5c786151783ce7d6be5",
"build_timestamp" : "2015-10-22T08:09:48Z",
"build_snapshot" : false,
"lucene_version" : "5.2.1"
},
"tagline" : "You Know, for Search"
}
```
As I said, both nodes are absolutely identical, because built from same docker image.
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [November 14, 2015, 5:48pm UTC](https://discuss.elastic.co/t/invalid-internal-transport-message-format-ff-f4-ff-fd/34586/4 "2015-11-14T17:48:05Z")
That is strange. Any chance you could run it outside docker context so we make sure docker does not interfere here?
What does your docker file look like?
---
### Author:  [@vladmiller](https://discuss.elastic.co/u/vladmiller)
#### Post date: [November 14, 2015, 6:28pm UTC](https://discuss.elastic.co/t/invalid-internal-transport-message-format-ff-f4-ff-fd/34586/5 "2015-11-14T18:28:22Z")
Dockerfile is pretty straightforward
```
FROM elasticsearch:2
RUN apt-get update && apt-get upgrade && apt-get install telnet
# RUN mkdir /scripts
ADD config/ /etc/elasticsearch
EXPOSE 9200 9300
USER elasticsearch
ENTRYPOINT ["elasticsearch", "--path.conf=/etc/elasticsearch"]
```
I tried to run same version from Dockerfile on my local mac computer and everything worked fine. I can try to use EBT java environment and setup es via **.ebextensions** , and I'm pretty sure this will work, however I wanted to solve this problem using Docker.
Just in case my elasticsearch config
```
cluster:
name: ${CLUSTER_NAME}
plugin.mandatory: cloud-aws
cloud.aws:
access_key: XXXXXXXXX
secret_key: YYYYYYYYY
region: us-east-1
discovery.type: ec2
discovery.ec2.ping_timeout: 30s
discovery.ec2.tag.Name: ${EC2_TAG_NAME}
discovery.ec2.host_type: private_dns
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping_timeout: 30s
discovery.zen.ping.unicast:
host: []
network.host: ${HOSTNAME}
network.publish_host: 255.255.255.0
http:
host: 0.0.0.0
compression: true
cors:
enabled: true
allow-origin: '*'
```
Maybe, if you're more familiar with ES code you can point to the place where ES generates requests to other nodes, so I can try to debug the issue further.
---
### Author:  [@vladmiller](https://discuss.elastic.co/u/vladmiller)
#### Post date: [November 16, 2015, 8:17am UTC](https://discuss.elastic.co/t/invalid-internal-transport-message-format-ff-f4-ff-fd/34586/6 "2015-11-16T08:17:23Z")
Hey David,
So I created new deployment without Docker and it still fails on EBT. I'll update this message with more information a bit later.
---
### Author:  [@vladmiller](https://discuss.elastic.co/u/vladmiller)
#### Post date: [November 19, 2015, 9:06am UTC](https://discuss.elastic.co/t/invalid-internal-transport-message-format-ff-f4-ff-fd/34586/7 "2015-11-19T09:06:30Z")
The issue related with Docker setup. When I installed ES directly on EC2 instances nodes joined in cluster without any issues
---
### Author:  [@xkidro](https://discuss.elastic.co/u/xkidro)
#### Post date: [November 30, 2015, 12:41pm UTC](https://discuss.elastic.co/t/invalid-internal-transport-message-format-ff-f4-ff-fd/34586/8 "2015-11-30T12:41:04Z")
@vladmiller did you find a way to do it in docker ? I am also very interested in doing this on Elastic Beanstalk however I failed every time.
---
### Author:  [@vladmiller](https://discuss.elastic.co/u/vladmiller)
#### Post date: [November 30, 2015, 1:33pm UTC](https://discuss.elastic.co/t/invalid-internal-transport-message-format-ff-f4-ff-fd/34586/9 "2015-11-30T13:33:13Z")
Hi @xkidro
No I have not found the way how to do it in docker, however instead of using docker I use java-8 environment and download ES directly to the host machine instead of docker container.
---
### Author:  [@xkidro](https://discuss.elastic.co/u/xkidro)
#### Post date: [December 3, 2015, 8:21am UTC](https://discuss.elastic.co/t/invalid-internal-transport-message-format-ff-f4-ff-fd/34586/10 "2015-12-03T08:21:14Z")
Using EB extension I guess? ssh-ing to it every time would be weird for an automated environment like Elastic Beanstalk.
If this is true, do you mind sharing it ?
---
### Author:  [@vladmiller](https://discuss.elastic.co/u/vladmiller)
#### Post date: [December 3, 2015, 8:52am UTC](https://discuss.elastic.co/t/invalid-internal-transport-message-format-ff-f4-ff-fd/34586/11 "2015-12-03T08:52:46Z")
Yes, using ebextensions.
I will publish that on weekends and send you a link. Busy weekdays 🙂
---
### Author:  [@xkidro](https://discuss.elastic.co/u/xkidro)
#### Post date: [December 3, 2015, 9:20am UTC](https://discuss.elastic.co/t/invalid-internal-transport-message-format-ff-f4-ff-fd/34586/12 "2015-12-03T09:20:04Z")
@vladmiller that's awesome news, thanks!
---
### Author:  [@vladmiller](https://discuss.elastic.co/u/vladmiller)
#### Post date: [December 7, 2015, 7:13am UTC](https://discuss.elastic.co/t/invalid-internal-transport-message-format-ff-f4-ff-fd/34586/13 "2015-12-07T07:13:13Z")
@xkidro, apologies. Wife demanded my time on weekends 😄
Here is the url for repo with config [https://github.com/vladmiller/elasticsearch-beanstalk](https://github.com/vladmiller/elasticsearch-beanstalk)
If you want to improve it please do; I am not the best sys ops guy out there
---
### Author:  [@xkidro](https://discuss.elastic.co/u/xkidro)
#### Post date: [December 7, 2015, 1:16pm UTC](https://discuss.elastic.co/t/invalid-internal-transport-message-format-ff-f4-ff-fd/34586/14 "2015-12-07T13:16:37Z")
@vladmiller thanks a lot! I will test this out when I have time, I'm not a good one my self 😃
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 5, 2017, 11:33pm UTC](https://discuss.elastic.co/t/invalid-internal-transport-message-format-ff-f4-ff-fd/34586/15 "2017-07-05T23:33:01Z")