Indice Stats Collector Crashing

[dashwood]

Hi,

We are currently mass importing data with the PHP client using bulk indexing. However, we recently ran into a problem where the indice count would crash mid import and we had no clue how many records were being processed. At first the record count seems kind of slow than it just crashes.

How would we avoid something like this? This has happened 3 times, I've restarted Elasticsearch many times.

Thank You

[2017-03-21T01:05:20,128][ERROR][o.e.x.m.c.i.IndicesStatsCollector] [Master] collector [indices-stats-collector] timed out when collecting data
[2017-03-21T01:07:12,752][ERROR][o.e.x.m.c.i.IndicesStatsCollector] [Master] collector [indices-stats-collector] timed out when collecting data
[2017-03-21T01:09:44,111][ERROR][o.e.x.m.c.i.IndicesStatsCollector] [Master] collector [indices-stats-collector] timed out when collecting data
[2017-03-21T01:11:41,026][ERROR][o.e.x.m.c.i.IndicesStatsCollector] [Master] collector [indices-stats-collector] timed out when collecting data
[2017-03-21T01:20:25,053][ERROR][o.e.x.m.c.i.IndicesStatsCollector] [Master] collector [indices-stats-collector] timed out when collecting data
[2017-03-21T01:21:07,991][ERROR][o.e.x.m.c.i.IndicesStatsCollector] [Master] collector [indices-stats-collector] timed out when collecting data
[2017-03-21T01:31:05,625][ERROR][o.e.x.m.c.i.IndicesStatsCollector] [Master] collector [indices-stats-collector] timed out when collecting data

[tanguy]

Hi,

How many indices and shards do you have?

[dashwood]

We have 1 indice and 1 shard, we were doing heavy bulk importing. It's not crashing anymore. However, we think the cause was due to the mass amount of bulk importing we were doing through the api and at the speed we were going. Elastic is on 16gb of ram, while the total system memory is 32gb. The 16 gb is for the system and disk cache.