# ES Cluster stops responding after trying to load data **URL:** \ **Category:** Elasticsearch\ **Created:** [September 25, 2016, 2:42am UTC](https://discuss.elastic.co/t/es-cluster-stops-responding-after-trying-to-load-data/61454 "2016-09-25T02:42:14Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![cookandy](https://avatars.discourse-cdn.com/v4/letter/c/e495f1/32.png) [@cookandy](https://discuss.elastic.co/u/cookandy)\ **Post date:** [September 25, 2016, 2:42am UTC](https://discuss.elastic.co/t/es-cluster-stops-responding-after-trying-to-load-data/61454/1 "2016-09-25T02:42:14Z")
Hello, I am testing ES with a very simple 2-node cluster. Upon startup, everything looks good. Similarly, when I query the health of the cluster on either node, it returns green: **Node 1:** ``` > curl 'http://10.138.160.210:31972/_cluster/health?pretty' { "cluster_name" : "es-cluster", "status" : "green", "timed_out" : false, "number_of_nodes" : 2, "number_of_data_nodes" : 2, .... "active_shards_percent_as_number" : 100.0 } ``` **Node 2 returns the same**. However, as soon as I try to load sample data, it just hangs. I am loading data with: ``` curl -XPOST 'http://10.138.96.56:31212/bank/account/_bulk?pretty' --data-binary "@accounts.json" ``` But the curl just hangs indefinitely and never finishes. All I see in the ES logs during this data load is: ``` [2016-09-25 02:24:14,049][INFO][cluster.metadata] [elastic-search-31212] [bank] creating index, cause [auto(bulk api)], templates [], shards [5]/[1], mappings [account] [2016-09-25 02:24:14,724][INFO][cluster.routing.allocation] [elastic-search-31212] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[bank][4], [bank][4]] ...]). [2016-09-25 02:24:14,781][INFO][cluster.metadata] [elastic-search-31212] [bank] update_mapping [account] ``` And when I look at the cluster status, it's no longer green. The curl never completes and the cluster health never returns to green. Any ideas what is happening? I'm using ES `2.3` running in a docker container. Thanks. ---
**Author:** ![cookandy](https://avatars.discourse-cdn.com/v4/letter/c/e495f1/32.png) [@cookandy](https://discuss.elastic.co/u/cookandy)\ **Post date:** [September 25, 2016, 2:42am UTC](https://discuss.elastic.co/t/es-cluster-stops-responding-after-trying-to-load-data/61454/2 "2016-09-25T02:42:51Z")
Here are the startup logs (couldn't include them in the original post due to character limit): **Node 1 startup log** ``` [2016-09-25 02:20:56,246][WARN][bootstrap] unable to install syscall filter: seccomp unavailable: your kernel is buggy and you should upgrade [2016-09-25 02:20:56,411][INFO][node] [elastic-search-31972] version[2.3.5], pid[11], build[90f439f/2016-07-27T10:36:52Z] [2016-09-25 02:20:56,411][INFO][node] [elastic-search-31972] initializing ... [2016-09-25 02:20:56,965][INFO][plugins] [elastic-search-31972] modules [reindex, lang-expression, lang-groovy], plugins [], sites [] [2016-09-25 02:20:56,991][INFO][env] [elastic-search-31972] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/vda1)]], net usable_space [67.8gb], net total_space [78.6gb], spins? [possibly], types [ext4] [2016-09-25 02:20:56,992][INFO][env] [elastic-search-31972] heap size [990.7mb], compressed ordinary object pointers [true] [2016-09-25 02:20:58,956][INFO][node] [elastic-search-31972] initialized [2016-09-25 02:20:58,957][INFO][node] [elastic-search-31972] starting ... [2016-09-25 02:20:59,033][INFO][transport] [elastic-search-31972] publish_address {10.138.160.210:31973}, bound_addresses {[::]:9300} [2016-09-25 02:20:59,038][INFO][discovery] [elastic-search-31972] es-cluster/uA9nSCnnRxapmen24AaT5Q [2016-09-25 02:21:29,041][WARN][discovery] [elastic-search-31972] waited for 30s and no initial state was set by the discovery [2016-09-25 02:21:29,052][INFO][http] [elastic-search-31972] publish_address {10.138.160.210:9200}, bound_addresses {[::]:9200} [2016-09-25 02:21:29,052][INFO][node] [elastic-search-31972] started [2016-09-25 02:21:29,296][INFO][cluster.service] [elastic-search-31972] detected_master {elastic-search-31212}{Gw-EirQpTduPgnwM2Ll18A}{10.138.96.56}{10.138.96.56:31213}, added {{elastic-search-31212}{Gw-EirQpTduPgnwM2Ll18A}{10.138.96.56}{10.138.96.56:31213},}, reason: zen-disco-receive(from master [{elastic-search-31212}{Gw-EirQpTduPgnwM2Ll18A}{10.138.96.56}{10.138.96.56:31213}]) ``` **Node 2 startup log** ``` [2016-09-25 02:20:56,326][WARN][bootstrap] unable to install syscall filter: seccomp unavailable: your kernel is buggy and you should upgrade [2016-09-25 02:20:56,491][INFO][node] [elastic-search-31212] version[2.3.5], pid[10], build[90f439f/2016-07-27T10:36:52Z] [2016-09-25 02:20:56,492][INFO][node] [elastic-search-31212] initializing ... [2016-09-25 02:20:57,092][INFO][plugins] [elastic-search-31212] modules [reindex, lang-expression, lang-groovy], plugins [], sites [] [2016-09-25 02:20:57,120][INFO][env] [elastic-search-31212] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/vda1)]], net usable_space [67.9gb], net total_space [78.6gb], spins? [possibly], types [ext4] [2016-09-25 02:20:57,120][INFO][env] [elastic-search-31212] heap size [990.7mb], compressed ordinary object pointers [true] [2016-09-25 02:20:59,114][INFO][node] [elastic-search-31212] initialized [2016-09-25 02:20:59,115][INFO][node] [elastic-search-31212] starting ... [2016-09-25 02:20:59,220][INFO][transport] [elastic-search-31212] publish_address {10.138.96.56:31213}, bound_addresses {[::]:9300} [2016-09-25 02:20:59,225][INFO][discovery] [elastic-search-31212] es-cluster/Gw-EirQpTduPgnwM2Ll18A [2016-09-25 02:21:29,228][WARN][discovery] [elastic-search-31212] waited for 30s and no initial state was set by the discovery [2016-09-25 02:21:29,255][INFO][http] [elastic-search-31212] publish_address {10.138.96.56:9200}, bound_addresses {[::]:9200} [2016-09-25 02:21:29,255][INFO][node] [elastic-search-31212] started [2016-09-25 02:21:29,266][INFO][cluster.service] [elastic-search-31212] new_master {elastic-search-31212}{Gw-EirQpTduPgnwM2Ll18A}{10.138.96.56}{10.138.96.56:31213}, added {{elastic-search-31972}{uA9nSCnnRxapmen24AaT5Q}{10.138.160.210}{10.138.160.210:31973},}, reason: zen-disco-join(elected_as_master, [1] joins received) [2016-09-25 02:21:29,390][INFO][gateway] [elastic-search-31212] recovered [0] indices into cluster_state ``` ---
**Author:** ![kstaken](https://avatars.discourse-cdn.com/v4/letter/k/e68b1a/32.png) [@kstaken](https://discuss.elastic.co/u/kstaken)\ **Post date:** [September 26, 2016, 8:57pm UTC](https://discuss.elastic.co/t/es-cluster-stops-responding-after-trying-to-load-data/61454/3 "2016-09-26T20:57:12Z")
How big is that accounts.json file? Bulk requests need to be reasonably sized so if it's large you may have to break it up. Kimbro ---
**Author:** ![cookandy](https://avatars.discourse-cdn.com/v4/letter/c/e495f1/32.png) [@cookandy](https://discuss.elastic.co/u/cookandy)\ **Post date:** [September 26, 2016, 9:38pm UTC](https://discuss.elastic.co/t/es-cluster-stops-responding-after-trying-to-load-data/61454/4 "2016-09-26T21:38:31Z")
Hi @kstaken. The accounts.json file is just the sample one from the [ES website](https://www.elastic.co/guide/en/elasticsearch/reference/2.3/_exploring_your_data.html) and is only 245k in size. After further inspection, this appears to be a problem with IPSec. I'm guessing it has something to do with the MTU, as I am able to load smaller files without a problem. And the issue only occurs after building an ES cluster. I can load the accounts.json file with a single server, no problem. I have opened a Github Issue as I think this is a problem with the size of the packets going between the ES nodes. Even though I've configured iptables to force the MTU to 1460 (to account for IPSec overhead), I still have this problem. I've posted more details on the GH issue page here: > Any help you can provide would be awesome! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 10:17pm UTC](https://discuss.elastic.co/t/es-cluster-stops-responding-after-trying-to-load-data/61454/5 "2017-07-05T22:17:10Z")