I've been using the Logstash JMX input succesfully for some time to monitor up to 10 Java JMX enabled processes with 1 Logstash process and nb_thread set to 4 (default).
However, I've recently configured a Logstash process which I gave a total of 88 Java JMX enabled processes to monitor.
I immediately ran into the following error :
"The time taken to retrieve metrics is more important than the retrieve_interval time set. you must adapt nb_thread, retrieve_interval to the number of jvm/metrics you want to retrieve."
I've been increasing the nb_thread up until 50 now, but the error remains.
- Is 88 processes for one Logstash JMX (JMX + some filters + ES output) simply too much and should I be looking to have more Logstashes each handling less processes ?
- Is this a Logstash limit or could it be an ES limit ? How could I detect if ES is too slow picking up the data in this case ?
Thanks in advance for any help or suggestions.