# Network.host issue: elasticsearch.service: Main process exited, code=exited, status=78 **URL:** \ **Category:** Elasticsearch\ **Created:** [September 19, 2019, 11:25am UTC](https://discuss.elastic.co/t/network-host-issue-elasticsearch-service-main-process-exited-code-exited-status-78/200193 "2019-09-19T11:25:12Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![the1bit](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/the1bit/32/54500_2.png) [@the1bit](https://discuss.elastic.co/u/the1bit)\ **Post date:** [September 19, 2019, 11:25am UTC](https://discuss.elastic.co/t/network-host-issue-elasticsearch-service-main-process-exited-code-exited-status-78/200193/1 "2019-09-19T11:25:12Z")
Hi, I try to configure an Elasticsearch cluster on Ubuntu 18.04 (on Azure). The basic installation is fine. Nevertheless when I try to configure the "network.host" value in the configuration file the Elasticsearch service does not start. Error message: > ## systemd[1]: Started Elasticsearch. -- Subject: Unit elasticsearch.service has finished start-up -- Defined-By: systemd -- Support: [Enterprise open source support | Ubuntu](http://www.ubuntu.com/support) > > ## -- Unit elasticsearch.service has finished starting up. > > -- The start-up result is RESULT. > elasticsearch[3955]: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release. > systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE > systemd[1]: elasticsearch.service: Failed with result 'exit-code'. The following entries don't work: network.host: 0.0.0.0 network.host: _eth0_ network.host: 10.79.10.17 - Elasticsearch version: 7.3.1 - JVM version: OpenJDK Runtime Environment (build 1.8.0\_222-8u222-b10-1ubuntu1~18.04.1-b10) - OS version: Ubuntu 18.04 LTS - vm.max\_map\_count=524288 (it does not work neither with default nor with 262144) By default it refuse the connection for static ip curl [http://10.79.10.17:9200](http://10.79.10.17:9200) curl: (7) Failed to connect to 10.79.10.17 port 9200: Connection refused When I try to configure network.host value the services does not start. Thank you for your help in this matter ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [September 19, 2019, 1:06pm UTC](https://discuss.elastic.co/t/network-host-issue-elasticsearch-service-main-process-exited-code-exited-status-78/200193/2 "2019-09-19T13:06:46Z")
can you check the logfile in `/var/log/elasticsearch` that is named like your cluster name is configured? I assume, that a bootstrap check has failed and that the log file contains information how to fix this, but this is just an assumption for now. See [https://www.elastic.co/guide/en/elasticsearch/reference/7.3/starting-elasticsearch.html#start-es-deb-systemd](https://www.elastic.co/guide/en/elasticsearch/reference/7.3/starting-elasticsearch.html#start-es-deb-systemd) ---
**Author:** ![the1bit](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/the1bit/32/54500_2.png) [@the1bit](https://discuss.elastic.co/u/the1bit)\ **Post date:** [September 19, 2019, 2:28pm UTC](https://discuss.elastic.co/t/network-host-issue-elasticsearch-service-main-process-exited-code-exited-status-78/200193/3 "2019-09-19T14:28:18Z")
> [@spinscale](#): > > `/var/log/elasticsearch` My config file: # Use a descriptive name for your cluster: # cluster.name: my-search # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # node.name: ${HOSTNAME} node.ingest: true node.data: false node.master: false # # Add custom attributes to the node: # #node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # path.data: /var/lib/elasticsearch # # Path to log files: # path.logs: /var/log/elasticsearch # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # #bootstrap.memory\_lock: true # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: 0.0.0.0 # # Set a custom port for HTTP: # ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [September 24, 2019, 8:23am UTC](https://discuss.elastic.co/t/network-host-issue-elasticsearch-service-main-process-exited-code-exited-status-78/200193/4 "2019-09-24T08:23:59Z")
this is the configuration file from `/etc/elasticsearch`, not the log file ---
**Author:** ![the1bit](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/the1bit/32/54500_2.png) [@the1bit](https://discuss.elastic.co/u/the1bit)\ **Post date:** [September 24, 2019, 1:01pm UTC](https://discuss.elastic.co/t/network-host-issue-elasticsearch-service-main-process-exited-code-exited-status-78/200193/5 "2019-09-24T13:01:37Z")
[/var/log/elasticsearch/gc.log](https://the1bithusacommon.blob.core.windows.net/public/gc.log) I can see this in the log: Entering safepoint region: GenCollectForAllocation GC(2) Pause Young (Allocation Failure) GC(2) Using 8 workers of 8 for evacuation GC(2) Desired survivor size 17891328 bytes, new threshold 6 (max threshold 6) GC(2) Age table with threshold 6 (max threshold 6) GC(2) - age 1: 2505304 bytes, 2505304 total GC(2) - age 2: 4066352 bytes, 6571656 total GC(2) - age 3: 10031824 bytes, 16603480 total GC(2) ParNew: 298211K-\>23277K(314560K) GC(2) CMS: 0K-\>0K(699072K) GC(2) Metaspace: 20810K-\>20810K(1069056K) GC(2) Pause Young (Allocation Failure) 291M-\>22M(989M) 5.505ms GC(2) User=0.04s Sys=0.00s Real=0.01s Leaving safepoint region Total time for which application threads were stopped: 0.0057126 seconds, Stopping threads took: 0.0000459 seconds ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [September 24, 2019, 1:19pm UTC](https://discuss.elastic.co/t/network-host-issue-elasticsearch-service-main-process-exited-code-exited-status-78/200193/6 "2019-09-24T13:19:47Z")
as written above, there should be a file, that is named like your cluster name, thus `my-search.log` in the log directory, please show the output of that one. ---
**Author:** ![the1bit](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/the1bit/32/54500_2.png) [@the1bit](https://discuss.elastic.co/u/the1bit)\ **Post date:** [September 24, 2019, 1:38pm UTC](https://discuss.elastic.co/t/network-host-issue-elasticsearch-service-main-process-exited-code-exited-status-78/200193/7 "2019-09-24T13:38:28Z")
You are right. I am not so familiar with elasticsearch logging. Here is the [my-search.log](https://the1bithusacommon.blob.core.windows.net/public/my-search.log) Thanks again ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [September 24, 2019, 1:49pm UTC](https://discuss.elastic.co/t/network-host-issue-elasticsearch-service-main-process-exited-code-exited-status-78/200193/8 "2019-09-24T13:49:18Z")
Perhaps take a look at [this](https://discuss.elastic.co/t/n00b-trying-to-set-up-on-windows/197514) post.... May explain why elasticsearch fails when you try to start it when you change `network.host` ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [September 25, 2019, 11:33am UTC](https://discuss.elastic.co/t/network-host-issue-elasticsearch-service-main-process-exited-code-exited-status-78/200193/9 "2019-09-25T11:33:42Z")
See this snippet from the logfile ```auto [2019-09-24T13:35:48,085][INFO][o.e.b.BootstrapChecks] [eslnx02] bound or publishing to a non-loopback address, enforcing bootstrap checks [2019-09-24T13:35:48,089][ERROR][o.e.b.Bootstrap] [eslnx02] node validation exception [3] bootstrap checks failed [1]: initial heap size [2147483648] not equal to maximum heap size [4294967296]; this can cause resize pauses and prevents mlockall from locking the entire heap [2]: memory locking requested for elasticsearch process but memory is not locked [3]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured ``` See [https://www.elastic.co/guide/en/elasticsearch/reference/7.3/bootstrap-checks.html](https://www.elastic.co/guide/en/elasticsearch/reference/7.3/bootstrap-checks.html) ---
**Author:** ![the1bit](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/the1bit/32/54500_2.png) [@the1bit](https://discuss.elastic.co/u/the1bit)\ **Post date:** [September 26, 2019, 10:27am UTC](https://discuss.elastic.co/t/network-host-issue-elasticsearch-service-main-process-exited-code-exited-status-78/200193/10 "2019-09-26T10:27:55Z")
I have fount the root cause of the issue: - If you configure the **network.host** value you also have to configure the **transport.host**. This means you have to add the following section to **elasticsearch.yml** and your system will work. ```auto network.host: 0.0.0.0 http.port: 9200 transport.host: _site_ transport.tcp.port: 9300 ``` Then you should restart the service then it works. 🙂 Additionally if you would like to enforce elasticsearch to listens on IPv4 sou have to add the next line to **/etc/elasticsearch/jvm.options** : ```auto -Djava.net.preferIPv4Stack=true ``` So my environment works fine now. 🙂 My comment: The related document does not mention this. 😥😢 Thank you for your support and your suggestions. Have a great day. ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [September 26, 2019, 11:43am UTC](https://discuss.elastic.co/t/network-host-issue-elasticsearch-service-main-process-exited-code-exited-status-78/200193/11 "2019-09-26T11:43:14Z")
> [@the1bit](#): > > - If you configure the **network.host** value you also have to configure the **transport.host**. I don't think that's true, and I have tried and confirmed that there is no problem running Elasticsearch with `network.host` set and `transport.host` unset. The issue you had was that there were bootstrap checks failing. ---
**Author:** ![the1bit](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/the1bit/32/54500_2.png) [@the1bit](https://discuss.elastic.co/u/the1bit)\ **Post date:** [September 26, 2019, 11:45am UTC](https://discuss.elastic.co/t/network-host-issue-elasticsearch-service-main-process-exited-code-exited-status-78/200193/12 "2019-09-26T11:45:17Z")
Thank you for your feedback. Nevertheless the bootstrap chack was ok and my solution was the key. This solved the same issue on my Windows host as well. ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [September 26, 2019, 11:46am UTC](https://discuss.elastic.co/t/network-host-issue-elasticsearch-service-main-process-exited-code-exited-status-78/200193/13 "2019-09-26T11:46:34Z")
Can you help us understand how to reproduce the problem you solved by setting `transport.host`? Normally this setting should not be set. ---
**Author:** ![the1bit](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/the1bit/32/54500_2.png) [@the1bit](https://discuss.elastic.co/u/the1bit)\ **Post date:** [September 26, 2019, 11:51am UTC](https://discuss.elastic.co/t/network-host-issue-elasticsearch-service-main-process-exited-code-exited-status-78/200193/14 "2019-09-26T11:51:18Z")
Sure. 🙂 If you merely configure this: ```auto network.host: 0.0.0.0 http.port: 9200 ``` You are facing the issue you can see above. Then if you expand your config file with the transport.host related configuration, the system works fine. ```auto transport.host: _site_ transport.tcp.port: 9300 ``` All together is the solution: ```auto network.host: 0.0.0.0 http.port: 9200 transport.host: _site_ transport.tcp.port: 9300 ``` My working **elasticsearch.yml** configuration now: ```auto cluster.name: my-search node.name: ${HOSTNAME} node.data: false node.master: true node.ingest: false path.data: /var/lib/elasticsearch path.logs: /var/log/elasticsearch network.host: 0.0.0.0 http.port: 9200 transport.host: _site_ transport.tcp.port: 9300 ``` Response of get request on port 9200: ```auto { "name": "eslnx02", "cluster_name": "my-search", "cluster_uuid": "tpIiL4e5QTicAdzspRM-GQ", "version": { "number": "7.3.2", "build_flavor": "default", "build_type": "deb", "build_hash": "1c1faf1", "build_date": "2019-09-06T14:40:30.409026Z", "build_snapshot": false, "lucene_version": "8.1.0", "minimum_wire_compatibility_version": "6.8.0", "minimum_index_compatibility_version": "6.0.0-beta1" }, "tagline": "You Know, for Search" } ``` ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [September 26, 2019, 11:57am UTC](https://discuss.elastic.co/t/network-host-issue-elasticsearch-service-main-process-exited-code-exited-status-78/200193/15 "2019-09-26T11:57:33Z")
When your node starts up, it logs two lines containing the string `bound_addresses` that look like this: ```auto [2019-09-26T12:41:10,124][INFO][o.e.t.TransportService] [node-0] publish_address {192.168.1.139:9300}, bound_addresses {192.168.1.139:9300}, {192.168.1.179:9300} ``` and ```auto [2019-09-26T12:41:10,787][INFO][o.e.h.AbstractHttpServerTransport] [node-0] publish_address {192.168.1.139:9200}, bound_addresses {192.168.1.139:9200}, {192.168.1.179:9200} ``` These might be logged some time apart. Can you share these lines here? ---
**Author:** ![the1bit](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/the1bit/32/54500_2.png) [@the1bit](https://discuss.elastic.co/u/the1bit)\ **Post date:** [September 26, 2019, 12:01pm UTC](https://discuss.elastic.co/t/network-host-issue-elasticsearch-service-main-process-exited-code-exited-status-78/200193/16 "2019-09-26T12:01:11Z")
Absolutely ```auto [2019-09-26T10:09:39,513][INFO][o.e.t.TransportService] [eslnx02] publish_address {10.10.10.21:9300}, bound_addresses {10.10.10.21:9300} [2019-09-26T10:09:39,789][INFO][o.e.h.AbstractHttpServerTransport] [eslnx02] publish_address {10.10.10.21:9200}, bound_addresses {0.0.0.0:9200} ``` ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [September 26, 2019, 12:12pm UTC](https://discuss.elastic.co/t/network-host-issue-elasticsearch-service-main-process-exited-code-exited-status-78/200193/17 "2019-09-26T12:12:14Z")
Thanks, I am wondering if there's a bug here. Do you see a line saying `bound or publishing to a non-loopback address` shortly after the first of these? I.e.: ```auto [2019-09-26T13:04:25,325][INFO][o.e.t.TransportService] [node-0] publish_address {192.168.1.139:9302}, bound_addresses {192.168.1.139:9302}, {192.168.1.179:9302} [2019-09-26T13:04:25,337][INFO][o.e.b.BootstrapChecks] [node-0] bound or publishing to a non-loopback address, enforcing bootstrap checks ``` Would it be possible to share the whole log output from starting up the node through to seeing the first `master node changed` message? Also, could you comment out just the `transport.*` lines in your config, restart the node and share the whole log output from starting up the node through to the whole message saying `bootstrap checks failed`? ---
**Author:** ![the1bit](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/the1bit/32/54500_2.png) [@the1bit](https://discuss.elastic.co/u/the1bit)\ **Post date:** [September 26, 2019, 1:22pm UTC](https://discuss.elastic.co/t/network-host-issue-elasticsearch-service-main-process-exited-code-exited-status-78/200193/18 "2019-09-26T13:22:43Z")
🙂 **bootstrap check** related row is there imediately after`publish_address {10.10.10.21:9300}, bound_addresses {10.10.10.21:9300}` : ```auto [2019-09-26T13:10:33,104][INFO][o.e.b.BootstrapChecks] [eslnx02] bound or publishing to a non-loopback address, enforcing bootstrap checks ``` **my-search.log** from **starting...** entry: ```auto [2019-09-26T13:10:32,850][INFO][o.e.n.Node] [eslnx02] starting ... [2019-09-26T13:10:33,097][INFO][o.e.t.TransportService] [eslnx02] publish_address {10.10.10.21:9300}, bound_addresses {10.10.10.21:9300} [2019-09-26T13:10:33,104][INFO][o.e.b.BootstrapChecks] [eslnx02] bound or publishing to a non-loopback address, enforcing bootstrap checks [2019-09-26T13:10:33,130][INFO][o.e.c.c.Coordinator] [eslnx02] cluster UUID [tpIiL4e5QTicAdzspRM-GQ] [2019-09-26T13:10:33,230][INFO][o.e.c.s.MasterService] [eslnx02] elected-as-master ([1] nodes joined)[{eslnx02}{DVNKx4n-QVuQbZwTvWdXDg}{ukvBIwpRSx28Sa1Ex9kzhw}{10.10.10.21}{10.10.10.21:9300}{m}{ml.machine_memory=33714167808, xpack.installed=true, ml.max_open_jobs=20} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 8, version: 37, reason: master node changed {previous [], current [{eslnx02}{DVNKx4n-QVuQbZwTvWdXDg}{ukvBIwpRSx28Sa1Ex9kzhw}{10.10.10.21}{10.10.10.21:9300}{m}{ml.machine_memory=33714167808, xpack.installed=true, ml.max_open_jobs=20}]} [2019-09-26T13:10:33,277][INFO][o.e.c.s.ClusterApplierService] [eslnx02] master node changed {previous [], current [{eslnx02}{DVNKx4n-QVuQbZwTvWdXDg}{ukvBIwpRSx28Sa1Ex9kzhw}{10.10.10.21}{10.10.10.21:9300}{m}{ml.machine_memory=33714167808, xpack.installed=true, ml.max_open_jobs=20}]}, term: 8, version: 37, reason: Publication{term=8, version=37} [2019-09-26T13:10:33,330][INFO][o.e.h.AbstractHttpServerTransport] [eslnx02] publish_address {10.10.10.21:9200}, bound_addresses {0.0.0.0:9200} [2019-09-26T13:10:33,330][INFO][o.e.n.Node] [eslnx02] started [2019-09-26T13:10:33,486][INFO][o.e.l.LicenseService] [eslnx02] license [50abffd9-f659-4aa2-913b-dd81223921aa] mode [basic] - valid [2019-09-26T13:10:33,487][INFO][o.e.x.s.s.SecurityStatusChangeListener] [eslnx02] Active license is now [BASIC]; Security is disabled [2019-09-26T13:10:33,495][INFO][o.e.g.GatewayService] [eslnx02] recovered [0] indices into cluster_state ``` ---
**Author:** ![the1bit](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/the1bit/32/54500_2.png) [@the1bit](https://discuss.elastic.co/u/the1bit)\ **Post date:** [September 26, 2019, 1:30pm UTC](https://discuss.elastic.co/t/network-host-issue-elasticsearch-service-main-process-exited-code-exited-status-78/200193/19 "2019-09-26T13:30:32Z")
Bootsrtap check failed: ```auto [1] bootstrap checks failed [1]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured ``` This means one of them is required from **discovery.seed\_hosts, discovery.seed\_providers, cluster.initial\_master\_nodes** Could you please send me the documentation where I can find this? 😉 ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [September 26, 2019, 1:36pm UTC](https://discuss.elastic.co/t/network-host-issue-elasticsearch-service-main-process-exited-code-exited-status-78/200193/20 "2019-09-26T13:36:37Z")
That specific check is documented [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/_discovery_configuration_check.html). However I really would like to see the complete logs from your node, both from when it successfully starts up and when it fails. The excerpts you've shared are unfortunately not enough for us to work out whether there's a bug that needs fixing here. [Next page](https://discuss.elastic.co/t/network-host-issue-elasticsearch-service-main-process-exited-code-exited-status-78/200193.md?page=2)