How to Capture a Heap Dump from a Running JVM - Logstash

This article applies to any product that runs a JVM -- Elasticsearch and Logstash 1.5 (or an earlier variant when using JRuby, which Logstash 1.5 uses).

NOTE - the JVM will pause during the collection of this heap dump, which means that Elasticsearch or Logstash will also pause. The duration of this pause depends on your heap size and the use of that heap.

To extract a heap dump from a running process we need to first find the Process ID (PID).

On Linux, run the following:

$ ps -ef | egrep -i 'elasticsearch|logstash' | grep -v grep

The PID will be the first number in the output.

On Windows, you can find the Process ID by using the following PowerShell command:

PS C:> Get-WmiObject Win32_Process -Filter "CommandLine like '%elasticsearch%' OR CommandLine like '%logstash%'" | select ProcessId,CommandLine

Alternatively, if you have access to a graphical console, you can use the Task Manager.

If you are running Elasticsearch as a Windows Service, then you use the Process ID of the service.

Then, to capture the heap dump run:

$ jmap -dump:format=b,file=heap.hprof

Use the jmap binary from the same Java installation and run jmap using the same user that was also used to start up the process. This ensures that jmap can access the running heap directly.

Once that is done, compress the output, either as a zip or tar.gz, and then send to support if requested.

Here is an example on Linux:

$ ps -ef|egrep 'elasticsearch|logstash' | grep -v grep

elastic+ 1667 1 2 06:56 ? 00:00:08 /usr/bin/java -Xms256m -Xmx1g -Xss256k -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Delasticsearch -Des.pidfile=/var/run/elasticsearch.pid -Des.path.home=/usr/share/elasticsearch -cp :/usr/share/elasticsearch/lib/elasticsearch-1.4.2.jar:/usr/share/elasticsearch/lib/:/usr/share/elasticsearch/lib/sigar/ -Des.default.config=/etc/elasticsearch/elasticsearch.yml -Des.default.path.home=/usr/share/elasticsearch -Des.default.path.logs=/var/log/elasticsearch -Des.default.path.data=/var/lib/elasticsearch -Des.default.path.work=/tmp/elasticsearch -Des.default.path.conf=/etc/elasticsearch org.elasticsearch.bootstrap.Elasticsearch

$ /usr/bin/jmap -dump:format=b,file=heap.hprof 1667

4 Likes