# Es don't index in 1-2 min? because of merge? or gc？ **URL:** \ **Category:** Elasticsearch\ **Created:** [August 7, 2013, 5:22pm UTC](https://discuss.elastic.co/t/es-dont-index-in-1-2-min-because-of-merge-or-gc/13109 "2013-08-07T17:22:19Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![baniu\_yao](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/baniu_yao/32/2168_2.png) [@baniu\_yao](https://discuss.elastic.co/u/baniu_yao)\ **Post date:** [August 7, 2013, 5:22pm UTC](https://discuss.elastic.co/t/es-dont-index-in-1-2-min-because-of-merge-or-gc/13109/1 "2013-08-07T17:22:19Z")
hey guys, I'm using flume(tool of apache to transfer big log data) to load data to elasticsearch in java. The problem I met is that the flume sometimes didn't move data to elasticsearch. For research, I try to measure the index number per second by calculating the delta `index_total` of the elasticsearch cluster by RESTful API. Well, I found that the delta index\_total equaled zero in one or two minutes, and, so that the flume hold the data which should be given to elasticsearch. After the `stop time` of elasticsearch, flume will handle the delayed data. A simple brief conclusion is, the `stop time` makes flume unstable. My question is, is the `stop time` usual? is that cuased by merge segments? or the Full GC? For Full GC, I remembered that elasticsearch will `stop the world` while full gc. And on the basis of this, we must make Xmx and Xms the same. But I found my elasticsearch node DO Full GC in about one hour, and the node works well. I'm not sure how can we do to elasticsearch except set Xms the same value as Xmx -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![baniu\_yao](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/baniu_yao/32/2168_2.png) [@baniu\_yao](https://discuss.elastic.co/u/baniu_yao)\ **Post date:** [August 8, 2013, 2:39am UTC](https://discuss.elastic.co/t/es-dont-index-in-1-2-min-because-of-merge-or-gc/13109/2 "2013-08-08T02:39:30Z")
some comments: I write a script to get increase index per second, and below is the result. What made me confused is why the increment of index is smooth? Is that because of some feature of elasticsearch for the sake of parameters? while i observed the delta index, the disk io of elasticsearch is very low when the delta index is zero. For my opinion, I think the data didn't not really hit elasticsearch. But, the flume(you can figure it out on my origin post) which measure the data flow speed told me that the data didn't go through to elasticsearch. It is odd. Thu Aug 8 10:34:59 2013 , 25819 Thu Aug 8 10:35:01 2013 , 36627 Thu Aug 8 10:35:03 2013 , 15181 Thu Aug 8 10:35:05 2013 , 33000 Thu Aug 8 10:35:07 2013 , 8358 Thu Aug 8 10:35:09 2013 , 8644 Thu Aug 8 10:35:11 2013 , 0 Thu Aug 8 10:35:13 2013 , 0 Thu Aug 8 10:35:16 2013 , 0 Thu Aug 8 10:35:18 2013 , 0 Thu Aug 8 10:35:20 2013 , 0 Thu Aug 8 10:35:22 2013 , 0 Thu Aug 8 10:35:24 2013 , 28884 Thu Aug 8 10:35:26 2013 , 36 Thu Aug 8 10:35:28 2013 , 0 Thu Aug 8 10:35:30 2013 , 0 Thu Aug 8 10:35:33 2013 , 0 Thu Aug 8 10:35:35 2013 , 0 Thu Aug 8 10:35:37 2013 , 0 Thu Aug 8 10:35:39 2013 , 0 Thu Aug 8 10:35:41 2013 , 41036 Thu Aug 8 10:35:43 2013 , 35360 Thu Aug 8 10:35:45 2013 , 25924 Thu Aug 8 10:35:47 2013 , 25589 Thu Aug 8 10:35:49 2013 , 17866 Thu Aug 8 10:35:51 2013 , 11310 Thu Aug 8 10:35:54 2013 , 8135 Thu Aug 8 10:35:56 2013 , 28000 Thu Aug 8 10:35:58 2013 , 19044 Thu Aug 8 10:36:00 2013 , 0 Thu Aug 8 10:36:02 2013 , 0 Thu Aug 8 10:36:04 2013 , 0 Thu Aug 8 10:36:06 2013 , 0 Thu Aug 8 10:36:08 2013 , 0 Thu Aug 8 10:36:10 2013 , 0 在 2013年8月8日星期四UTC+8上午1时22分19秒，姚仁捷写道： > hey guys, > > I'm using flume(tool of apache to transfer big log data) to load data to > elasticsearch in java. The problem I met is that the flume sometimes didn't > move data to elasticsearch. For research, I try to measure the index > number per second by calculating the delta `index_total` of the > elasticsearch cluster by RESTful API. > > Well, I found that the delta index\_total equaled zero in one or two > minutes, and, so that the flume hold the data which should be given to > elasticsearch. After the `stop time` of elasticsearch, flume will handle > the delayed data. > > A simple brief conclusion is, the `stop time` makes flume unstable. > > My question is, is the `stop time` usual? is that cuased by merge > segments? or the Full GC? > > For Full GC, I remembered that elasticsearch will `stop the world` while > full gc. And on the basis of this, we must make Xmx and Xms the same. But I > found my elasticsearch node DO Full GC in about one hour, and the node > works well. I'm not sure how can we do to elasticsearch except set Xms the > same value as Xmx -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:22am UTC](https://discuss.elastic.co/t/es-dont-index-in-1-2-min-because-of-merge-or-gc/13109/3 "2017-07-06T02:22:15Z")
Closed on July 6, 2017, 2:22am UTC