# Write and flush on the network layer failed

**URL:** <https://discuss.elastic.co/t/write-and-flush-on-the-network-layer-failed/119897>\
**Category:** Elasticsearch\
**Created:** [February 15, 2018, 1:54am UTC](https://discuss.elastic.co/t/write-and-flush-on-the-network-layer-failed/119897 "2018-02-15T01:54:30Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![bladerunner512](https://avatars.discourse-cdn.com/v4/letter/b/b77776/32.png) [@bladerunner512](https://discuss.elastic.co/u/bladerunner512)\
**Post date:** [February 15, 2018, 1:54am UTC](https://discuss.elastic.co/t/write-and-flush-on-the-network-layer-failed/119897/1 "2018-02-15T01:54:30Z")

</div>

Unable to get Docker ELK cluster working with 2 hosts. Both hosts have same response:

user01@dcw0970:~/docker-elk-master$ docker logs dockerelkmaster\_elasticsearch\_1  
[2018-02-15T01:33:13,054][INFO][o.e.n.Node] [896e8523da84] initializing ...  
[2018-02-15T01:33:13,123][INFO][o.e.e.NodeEnvironment] [896e8523da84] using [1] data paths, mounts [[/ (none)]], net usable\_space [5.2gb], net total\_space [7.3gb], types [aufs]  
[2018-02-15T01:33:13,123][INFO][o.e.e.NodeEnvironment] [896e8523da84] heap size [989.8mb], compressed ordinary object pointers [true]  
[2018-02-15T01:33:13,125][INFO][o.e.n.Node] [896e8523da84] node name [896e8523da84], node ID [lXvb0g0PT1qKX9i7h8DvcA]  
[2018-02-15T01:33:13,125][INFO][o.e.n.Node] [896e8523da84] version[6.0.0], pid[1], build[8f0685b/2017-11-10T18:41:22.859Z], OS[Linux/4.4.0-98-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0\_151/25.151-b12]  
[2018-02-15T01:33:13,125][INFO][o.e.n.Node] [896e8523da84] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -XX:+HeapDumpOnOutOfMemoryError, -Des.cgroups.hierarchy.override=/, -Xmx1g, -Xms1g, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config]  
[2018-02-15T01:33:13,839][INFO][o.e.p.PluginsService] [896e8523da84] loaded module [aggs-matrix-stats]  
[2018-02-15T01:33:13,839][INFO][o.e.p.PluginsService] [896e8523da84] loaded module [analysis-common]  
[2018-02-15T01:33:13,839][INFO][o.e.p.PluginsService] [896e8523da84] loaded module [ingest-common]  
[2018-02-15T01:33:13,839][INFO][o.e.p.PluginsService] [896e8523da84] loaded module [lang-expression]  
[2018-02-15T01:33:13,839][INFO][o.e.p.PluginsService] [896e8523da84] loaded module [lang-mustache]  
[2018-02-15T01:33:13,839][INFO][o.e.p.PluginsService] [896e8523da84] loaded module [lang-painless]  
[2018-02-15T01:33:13,839][INFO][o.e.p.PluginsService] [896e8523da84] loaded module [parent-join]  
[2018-02-15T01:33:13,840][INFO][o.e.p.PluginsService] [896e8523da84] loaded module [percolator]  
[2018-02-15T01:33:13,840][INFO][o.e.p.PluginsService] [896e8523da84] loaded module [reindex]  
[2018-02-15T01:33:13,840][INFO][o.e.p.PluginsService] [896e8523da84] loaded module [repository-url]  
[2018-02-15T01:33:13,840][INFO][o.e.p.PluginsService] [896e8523da84] loaded module [transport-netty4]  
[2018-02-15T01:33:13,840][INFO][o.e.p.PluginsService] [896e8523da84] loaded module [tribe]  
[2018-02-15T01:33:13,840][INFO][o.e.p.PluginsService] [896e8523da84] loaded plugin [ingest-geoip]  
[2018-02-15T01:33:13,840][INFO][o.e.p.PluginsService] [896e8523da84] loaded plugin [ingest-user-agent]  
[2018-02-15T01:33:14,877][INFO][o.e.d.DiscoveryModule] [896e8523da84] using discovery type [zen]  
[2018-02-15T01:33:15,331][INFO][o.e.n.Node] [896e8523da84] initialized  
[2018-02-15T01:33:15,331][INFO][o.e.n.Node] [896e8523da84] starting ...  
[2018-02-15T01:33:15,537][INFO][o.e.t.TransportService] [896e8523da84] publish\_address {xx.xxx.xx.248:9300}, bound\_addresses {0.0.0.0:9300}  
[2018-02-15T01:33:15,547][INFO][o.e.b.BootstrapChecks] [896e8523da84] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks  
[2018-02-15T01:33:15,622][WARN][o.e.t.n.Netty4Transport] [896e8523da84] write and flush on the network layer failed (channel: [id: 0xace1b77f, L:0.0.0.0/0.0.0.0:59142 ! R:dcw0970.dev.elk.com/xx.xxx.xx.248:9200])  
java.nio.channels.ClosedChannelException: null  
at io.netty.channel.AbstractChannel$AbstractUnsafe.write(...)(Unknown Source) ~[?:?]  
[2018-02-15T01:33:16,576][WARN][o.e.t.n.Netty4Transport] [896e8523da84] write and flush on the network layer failed (channel: [id: 0xa1107c7c, L:0.0.0.0/0.0.0.0:59148 ! R:dcw0970.dev.elk.com/xx.xxx.xx.248:9200])  
java.nio.channels.ClosedChannelException: null  
at io.netty.channel.AbstractChannel$AbstractUnsafe.write(...)(Unknown Source) ~[?:?]  
[2018-02-15T01:33:17,577][WARN][o.e.t.n.Netty4Transport] [896e8523da84] write and flush on the network layer failed (channel: [id: 0xb3e960d8, L:0.0.0.0/0.0.0.0:59156 ! R:dcw0970.dev.elk.com/xx.xxx.xx.248:9200])  
java.nio.channels.ClosedChannelException: null  
at io.netty.channel.AbstractChannel$AbstractUnsafe.write(...)(Unknown Source) ~[?:?]  
[2018-02-15T01:33:18,596][INFO][o.e.c.s.MasterService] [896e8523da84] zen-disco-elected-as-master ([0] nodes joined), reason: new\_master {896e8523da84}{lXvb0g0PT1qKX9i7h8DvcA}{i1QPfGm0SFi2kueAG9NBqw}{xx.xxx.xx.248}{xx.xxx.xx.248:9300}  
[2018-02-15T01:33:18,608][INFO][o.e.c.s.ClusterApplierService] [896e8523da84] new\_master {896e8523da84}{lXvb0g0PT1qKX9i7h8DvcA}{i1QPfGm0SFi2kueAG9NBqw}{xx.xxx.xx.248}{xx.xxx.xx.248:9300}, reason: apply cluster state (from master [master {896e8523da84}{lXvb0g0PT1qKX9i7h8DvcA}{i1QPfGm0SFi2kueAG9NBqw}{xx.xxx.xx.248}{xx.xxx.xx.248:9300} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]])  
[2018-02-15T01:33:18,621][INFO][o.e.h.n.Netty4HttpServerTransport] [896e8523da84] publish\_address {xx.xxx.xx.248:9200}, bound\_addresses {0.0.0.0:9200}  
[2018-02-15T01:33:18,621][INFO][o.e.n.Node] [896e8523da84] started  
[2018-02-15T01:33:18,628][INFO][o.e.g.GatewayService] [896e8523da84] recovered [0] indices into cluster\_state

elasticsearch.yml:

node.name: ${HOSTNAME}  
cluster.name: "docker-dev-elk"  
network.host: 0.0.0.0  
network.publish\_host: xx.xxx.xx.248

discovery.zen.minimum\_master\_nodes: 2  
gateway.expected\_nodes: 2  
gateway.recover\_after\_time: 5m  
gateway.recover\_after\_nodes: 2

discovery.zen.ping.unicast.hosts:  
-dcw0970.dev.elk.com:9200  
-dcw0971.dev.elk.com:9200

Both hosts have green status:  
{  
"cluster\_name": "docker-dev-elk",  
"status": "green",  
"timed\_out": false,  
"number\_of\_nodes": 1,  
"number\_of\_data\_nodes": 1,  
"active\_primary\_shards": 0,  
"active\_shards": 0,  
"relocating\_shards": 0,  
"initializing\_shards": 0,  
"unassigned\_shards": 0,  
"delayed\_unassigned\_shards": 0,  
"number\_of\_pending\_tasks": 0,  
"number\_of\_in\_flight\_fetch": 0,  
"task\_max\_waiting\_in\_queue\_millis": 0,  
"active\_shards\_percent\_as\_number": 100  
}

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [February 15, 2018, 6:23am UTC](https://discuss.elastic.co/t/write-and-flush-on-the-network-layer-failed/119897/2 "2018-02-15T06:23:27Z")

</div>

Elasticsearch use port 9300 to communicate between nodes, so you need to change the port number from 9200 to 9300 for unicast hosts in your config.

---

<div class="post-metadata">

**Author:** ![bladerunner512](https://avatars.discourse-cdn.com/v4/letter/b/b77776/32.png) [@bladerunner512](https://discuss.elastic.co/u/bladerunner512)\
**Post date:** [February 15, 2018, 3:05pm UTC](https://discuss.elastic.co/t/write-and-flush-on-the-network-layer-failed/119897/3 "2018-02-15T15:05:02Z")

</div>

Thanks. Not sure how I missed that, but works great.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 15, 2018, 3:05pm UTC](https://discuss.elastic.co/t/write-and-flush-on-the-network-layer-failed/119897/4 "2018-03-15T15:05:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
