# Data downfalls every 5 mins **URL:** \ **Category:** Elasticsearch\ **Created:** [January 6, 2017, 6:05pm UTC](https://discuss.elastic.co/t/data-downfalls-every-5-mins/70807 "2017-01-06T18:05:12Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![dtrizna](https://avatars.discourse-cdn.com/v4/letter/d/c2a13f/32.png) [@dtrizna](https://discuss.elastic.co/u/dtrizna)\ **Post date:** [January 6, 2017, 6:05pm UTC](https://discuss.elastic.co/t/data-downfalls-every-5-mins/70807/1 "2017-01-06T18:05:12Z")
Hi, everyone! I'm facing some weird Elastic behavior, and didn't found solution to something similar in the Web. There appear some data downfalls in about every 5 mins. The data itself is coming to box without such interrupts. After whole server reboot 2-3 hours documents are collected normally, but then the problem is again active. I have hourly indices. In beginning of every hour (while the next index is fresh?) there's some time interval, when data also is collected without such looses: ![](https://us1.discourse-cdn.com/elastic/original/2X/c/cc7c9eb92b51544ec4fd86aaebc9c560212ca6b3.png) Each index is apprx. 3 Gb per size, 60-70 Gb per day. Each index contains 3 shards (with lower number of shards there's 5-10% less documents per index). 30 s refresh interval, 0 replicas (one node). IO shouldn't be the bottle next, HDDs are located locally, 30-50 MB/s write speed, for buff/cache operations I dedicated 30 Gb of RAM, iowait is lower than 5%. CPU 10-12%. JMV heap is 30 GB, with 20% buffer size (some space there's reserved for parallel reindexation when it may be necessary). Average heap use isn't bigger than 60%. There isn't anything in logs. Only info about index creations, mapping updates.. So I'm really obsessed now and don't know where to look further. Probably someone was expecting something similar and know how to deal with it? Or any help with troubleshooting would be nice.. Thanks! ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [January 6, 2017, 10:48pm UTC](https://discuss.elastic.co/t/data-downfalls-every-5-mins/70807/2 "2017-01-06T22:48:05Z")
Given the regularity it could be GC? Do you have Monitoring installed? ---
**Author:** ![dtrizna](https://avatars.discourse-cdn.com/v4/letter/d/c2a13f/32.png) [@dtrizna](https://discuss.elastic.co/u/dtrizna)\ **Post date:** [January 7, 2017, 12:04am UTC](https://discuss.elastic.co/t/data-downfalls-every-5-mins/70807/3 "2017-01-07T00:04:09Z")
No, I've concentrated on data collection config, until now it was enough with API queries to get necessary monitoring info. Previously, when server had lower amount of memory, I've seen GC pauses (mentioned in logs). But now I don't observe that. ---
**Author:** ![dtrizna](https://avatars.discourse-cdn.com/v4/letter/d/c2a13f/32.png) [@dtrizna](https://discuss.elastic.co/u/dtrizna)\ **Post date:** [January 27, 2017, 12:38pm UTC](https://discuss.elastic.co/t/data-downfalls-every-5-mins/70807/4 "2017-01-27T12:38:10Z")
Actually there was a problem with sending application. I used ntopng instead of Logstash, and it wasn't capable to proceed such amount of requests. After I migrated to Logstash and tuned it, I received a huge productivity boost. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 24, 2017, 12:38pm UTC](https://discuss.elastic.co/t/data-downfalls-every-5-mins/70807/5 "2017-02-24T12:38:26Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.