What does this error mean?

Elastic Stack Logstash
1

I occasionally see errors that look like this in the logstash stdout:

Jun 04, 2015 8:48:01 AM org.elasticsearch.monitor.jvm.JvmMonitorService$JvmMonitor monitorLongGc
WARNING: [logstash-es1-20422-9784] [gc][young][219][293] duration [1.5s], collections [1]/[2.5s], total [1.5s]/[16.3s], memory [373.6mb]->[56.2mb]/[483.3mb], all_pools {[young] [103.7mb]->[868kb]/[133.3mb]}{[survivor] [16.6mb]->[0b]/[16.6mb]}{[old] [253.2mb]->[55.3mb]/[333.3mb]}

We're using this import for testing purposes to evaluate our server requirements - it's a single 16G machine (8GB heap) with almost 200M rows and +50GB of data stored generated from about 350GB of logs files (we have stored off for everything and using doc_fields) - perhaps these errors is a sign that we're hitting a limit?

2

It looks like the Garbage Collection on your Logstash instance took too long. In this case 1.5 seconds. You're probably right that you're hitting a limit.

3

FYI - I upped the logstash heap to 2G and I haven't seen the error any more.