# Huge data import from MySQL to ES failure

**URL:** <https://discuss.elastic.co/t/huge-data-import-from-mysql-to-es-failure/144517>\
**Category:** Elasticsearch\
**Created:** [August 15, 2018, 12:25pm UTC](https://discuss.elastic.co/t/huge-data-import-from-mysql-to-es-failure/144517 "2018-08-15T12:25:22Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![DarkTIm88](https://avatars.discourse-cdn.com/v4/letter/d/b19c9b/32.png) [@DarkTIm88](https://discuss.elastic.co/u/DarkTIm88)\
**Post date:** [August 15, 2018, 12:25pm UTC](https://discuss.elastic.co/t/huge-data-import-from-mysql-to-es-failure/144517/1 "2018-08-15T12:25:22Z")

</div>

Hello Everybody,

I'm new in ES, trying to import 32m rows from MySQL to Elasticsearch using Logstash. It works fine, but when the cluster reached 3.5m it breaks and doesn't allow to add more data. I've increased heap size to 2GB, disabled the refresh\_interval, but it still doesn't work. The hosting configuration is 4 Xeon CPU, 12GB RAM, ~200GB free on SSD.

The log show that the problem is allocation failure  
2018-08-15T05:14:42.867-0700: 4980.112: [GC (Allocation Failure) 2018-08-15T05:14:42.867-0700: 4980.112: [ParNew  
Desired survivor size 17432576 bytes, new threshold 1 (max 6)

And the cluster status is yellow, may it cause because I'm using a simple node?  
{  
"cluster\_name":"elasticsearch",  
"status":"yellow",  
"timed\_out":false,  
"number\_of\_nodes":1,  
"number\_of\_data\_nodes":1,  
"active\_primary\_shards":5,  
"active\_shards":5,  
"relocating\_shards":0,  
"initializing\_shards":0,  
"unassigned\_shards":5,  
"delayed\_unassigned\_shards":0  
number\_of\_pending\_tasks":0,  
"number\_of\_in\_flight\_fetch":0,  
"task\_max\_waiting\_in\_queue\_millis":0,  
"active\_shards\_percent\_as\_number":50.0  
}

Thank you for any advice!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 12, 2018, 12:39pm UTC](https://discuss.elastic.co/t/huge-data-import-from-mysql-to-es-failure/144517/2 "2018-09-12T12:39:13Z")

</div>

Closed on September 12, 2018, 12:39pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.