ES Crash

DJP · July 4, 2013, 11:14am

Hello,

I'm experiencing crash with ES about 1m30 to 30 min after ES starts with
differents versions.

I tried :

ES 19.9 + OpenJDK 1.6 ;
ES 19.12 + OpenJDK 1.6 ;
ES 19.12 + OracleJava 1.7 ;
ES 20.6 + OracleJava 1.7 ;
ES 90.2 + OracleJava 1.7 ;

And still crash. I flushed the database many times, and only few data is
stored in the instance. I run other ES single instance without any issue.

Here's crash info while running in foreground with the last configuration
(ES 90.2 + Java 1.7) :

A fatal error has been detected by the Java Runtime Environment:

SIGSEGV (0xb) at pc=0x00007fc375acf2cf, pid=10387, tid=140477327230720

JRE version: 7.0_25-b15

Java VM: Java HotSpot(TM) 64-Bit Server VM (23.25-b01 mixed mode

linux-amd64 compressed oops)

Problematic frame:

V [libjvm.so+0x3982cf] CodeCache::free(CodeBlob*)+0xf

Failed to write core dump. Core dumps have been disabled. To enable core

dumping, try "ulimit -c unlimited" before starting Java again

An error report file with more information is saved as:

/var/log/elasticsearch/hs_err_pid10387.log

If you would like to submit a bug report, please visit:

http://bugreport.sun.com/bugreport/crash.jsp

Aborted

And here's the full dump :

A fatal error has been detected by the Java Runtime Environment:

SIGSEGV (0xb) at pc=0x00007fc375acf2cf, pid=10387, tid=140477327230720

JRE version: 7.0_25-b15

Java VM: Java HotSpot(TM) 64-Bit Server VM (23.25-b01 mixed mode

linux-amd64 compressed oops)

Problematic frame:

V [libjvm.so+0x3982cf] CodeCache::free(CodeBlob*)+0xf

Failed to write core dump. Core dumps have been disabled. To enable core

dumping, try "ulimit -c unlimited" before starting Java again

If you would like to submit a bug report, please visit:

http://bugreport.sun.com/bugreport/crash.jsp

--------------- T H R E A D ---------------
Current thread (0x0000000001f0a800): JavaThread "C2 CompilerThread0"
daemon [_thread_in_vm, id=10400,
stack(0x00007fc36d204000,0x00007fc36d305000)]
siginfo:si_signo=SIGSEGV: si_errno=0, si_code=128 (),
si_addr=0x0000000000000000
Registers:
RAX=0x00007fc37151e140, RBX=0x00007fc37151a690, RCX=0x0000000000000000,
RDX=0x0000000000000000
RSP=0x00007fc36d302230, RBP=0x00007fc36d302240, RSI=0x0000000000000001,
RDI=0x00007fc37151a690
R8 =0x00007fc376426760, R9 =0x0000000000270000, R10=0x00007fc3767d8eb8,
R11=0x0000000000000030
R12=0x0000000001f0a800, R13=0x00007fc37151a690, R14=0x00000000024f9c10,
R15=0x00007fc3763ec5e0
RIP=0x00007fc375acf2cf, EFLAGS=0x0000000000010206,
CSGSFS=0x0000000000000033, ERR=0x0000000000000000
TRAPNO=0x000000000000000d
Top of Stack: (sp=0x00007fc36d302230)
0x00007fc36d302230: 0000000001df59f0 0000000001df59f0
0x00007fc36d302240: 00007fc36d302270 00007fc375aca846
0x00007fc36d302250: 00000000024f9c10 00007fc36d302e40
0x00007fc36d302260: 00000000024f5b90 00000000024fdb78
0x00007fc36d302270: 00007fc36d302290 00007fc375ae8160
0x00007fc36d302280: 00000000024f9c10 0000000000000000
0x00007fc36d302290: 00007fc36d302e80 00007fc375af009d
0x00007fc36d3022a0: 0000000000000008 0000000002522850
0x00007fc36d3022b0: 00007fc36d303740 00007fc36d303758
0x00007fc36d3022c0: 0000000001f0a310 0000000000000004
0x00007fc36d3022d0: 0000000000000000 0000000000000000
0x00007fc36d3022e0: 00007fc36d302ee0 00007fc36d302db0
0x00007fc36d3022f0: 00007fc36d302cd0 00007fc36d302e00
0x00007fc36d302300: 0000000001f0a310 00007fc36d303740
0x00007fc36d302310: 00007fc3763eaff0 0000000000000004
0x00007fc36d302320: 00007fc36d302ee0 00007fc36d303010
0x00007fc36d302330: 00007fc36d303010 0000000000000004
0x00007fc36d302340: 0000000001f95d50 00007fc36d303010
0x00007fc36d302350: 0000000000000100 00000000023fa430
0x00007fc36d302360: 0000000000000000 0000000000000000
0x00007fc36d302370: 0000000000000000 0000000000000000
0x00007fc36d302380: 0000000000000000 0000000000000000
0x00007fc36d302390: 0000000000000000 0000000000000000
0x00007fc36d3023a0: 0000000000000000 0000000000000000
0x00007fc36d3023b0: 0000000000000000 0000000000000000
0x00007fc36d3023c0: 0000000000000000 0000000000000000
0x00007fc36d3023d0: 0000000000000000 0000000000000000
0x00007fc36d3023e0: 0000000000000000 0000000000000000
0x00007fc36d3023f0: 0000000000000000 0000000000000000
0x00007fc36d302400: 0000000000000000 0000000000000000
0x00007fc36d302410: 0000000000000000 0000000000000000
0x00007fc36d302420: 0000000000000000 0000000000000000
Instructions: (pc=0x00007fc375acf2cf)
0x00007fc375acf2af: 8d 3c 03 29 c6 5b c9 e9 e5 bf 15 00 90 0f 1f 40
0x00007fc375acf2bf: 00 55 48 89 e5 53 48 89 fb 48 83 ec 08 48 8b 07
0x00007fc375acf2cf: ff 50 08 84 c0 74 28 83 2d 5b 10 95 00 01 48 63
0x00007fc375acf2df: 93 c0 00 00 00 48 63 83 bc 00 00 00 48 8d 14 13
Register to memory mapping:
RAX=0x00007fc37151e140 is an unknown value
RBX=0x00007fc37151a690 is an unknown value
RCX=0x0000000000000000 is an unknown value
RDX=0x0000000000000000 is an unknown value
RSP=0x00007fc36d302230 is pointing into the stack for thread:
0x0000000001f0a800
RBP=0x00007fc36d302240 is pointing into the stack for thread:
0x0000000001f0a800
RSI=0x0000000000000001 is an unknown value
RDI=0x00007fc37151a690 is an unknown value
R8 =0x00007fc376426760: <offset 0xcef760> in
/usr/lib/jvm/java-7-oracle/lib/amd64/server/libjvm.so at 0x00007fc375737000
R9 =0x0000000000270000 is an unknown value
R10=0x00007fc3767d8eb8: <offset 0x381eb8> in /lib/libc.so.6 at
0x00007fc376457000
R11=0x0000000000000030 is an unknown value
R12=0x0000000001f0a800 is a thread
R13=0x00007fc37151a690 is an unknown value
R14=0x00000000024f9c10 is an unknown value
R15=0x00007fc3763ec5e0: <offset 0xcb55e0> in
/usr/lib/jvm/java-7-oracle/lib/amd64/server/libjvm.so at 0x00007fc375737000

Stack: [0x00007fc36d204000,0x00007fc36d305000], sp=0x00007fc36d302230,
free space=1016k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
code)
V [libjvm.so+0x3982cf] CodeCache::free(CodeBlob*)+0xf
V [libjvm.so+0x393846] BufferBlob::free(BufferBlob*)+0x66
V [libjvm.so+0x3b1160] CompileWrapper::~CompileWrapper()+0x20
V [libjvm.so+0x3b909d] Compile::Compile(ciEnv*, C2Compiler*, ciMethod*,
int, bool, bool)+0xeed
V [libjvm.so+0x32ae52] C2Compiler::compile_method(ciEnv*, ciMethod*,
int)+0x142
V [libjvm.so+0x3bdee3]
CompileBroker::invoke_compiler_on_method(CompileTask*)+0x323
V [libjvm.so+0x3bf04d] CompileBroker::compiler_thread_loop()+0x43d
V [libjvm.so+0x861bc8] JavaThread::thread_main_inner()+0xc8
V [libjvm.so+0x861d18] JavaThread::run()+0x138
V [libjvm.so+0x744270] java_start(Thread*)+0x100

Current CompileTask:
C2: 97404 302
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask::compareTo
(9 bytes)

--------------- P R O C E S S ---------------
Java Threads: ( => current thread )
0x00000000024e1800 JavaThread "elasticsearch[Black Marvel][search][T#6]"
daemon [_thread_blocked, id=10428,
stack(0x00007fc367db7000,0x00007fc367df8000)]
0x00000000022c4800 JavaThread "elasticsearch[Black Marvel][search][T#5]"
daemon [_thread_blocked, id=10427,
stack(0x00007fc367df8000,0x00007fc367e39000)]
0x00000000022c3000 JavaThread "elasticsearch[Black Marvel][search][T#4]"
daemon [_thread_blocked, id=10426,
stack(0x00007fc367e39000,0x00007fc367e7a000)]
0x00000000022c2000 JavaThread "elasticsearch[Black Marvel][search][T#3]"
daemon [_thread_blocked, id=10425,
stack(0x00007fc367e7a000,0x00007fc367ebb000)]
0x00000000022a1800 JavaThread "elasticsearch[Black Marvel][search][T#2]"
daemon [_thread_blocked, id=10424,
stack(0x00007fc367ebb000,0x00007fc367efc000)]
0x0000000002297800 JavaThread "elasticsearch[Black Marvel][search][T#1]"
daemon [_thread_blocked, id=10423,
stack(0x00007fc367efc000,0x00007fc367f3d000)]
0x00007fc3686c9800 JavaThread "DestroyJavaVM" [_thread_blocked, id=10393,
stack(0x00007fc376fe6000,0x00007fc377027000)]
0x00007fc3686bf800 JavaThread "elasticsearch[keepAlive/0.20.6]"
[_thread_blocked, id=10422, stack(0x00007fc367f3d000,0x00007fc367f7e000)]
0x00007fc368240000 JavaThread "elasticsearch[Black
Marvel][http_server_boss][T#1]{New I/O server boss #9}" daemon
[_thread_in_native, id=10421, stack(0x00007fc367f7e000,0x00007fc367fbf000)]
0x00007fc368243800 JavaThread "elasticsearch[Black
Marvel][http_server_worker][T#2]{New I/O worker #8}" daemon
[_thread_in_native, id=10420, stack(0x00007fc367fbf000,0x00007fc368000000)]
0x00007fc368242000 JavaThread "elasticsearch[Black
Marvel][http_server_worker][T#1]{New I/O worker #7}" daemon
[_thread_in_native, id=10419, stack(0x00007fc36c034000,0x00007fc36c075000)]
0x0000000002205800 JavaThread "elasticsearch[Black
Marvel][riverClusterService#updateTask][T#1]" daemon [_thread_blocked,
id=10418, stack(0x00007fc36c075000,0x00007fc36c0b6000)]
0x00000000021fe800 JavaThread "elasticsearch[Black
Marvel][transport_client_timer][T#1]{Hashed wheel timer #1}" daemon
[_thread_blocked, id=10417, stack(0x00007fc36c0b6000,0x00007fc36c0f7000)]
0x0000000002234800 JavaThread "elasticsearch[Black
Marvel][clusterService#updateTask][T#1]" daemon [_thread_blocked, id=10416,
stack(0x00007fc36c0f7000,0x00007fc36c138000)]
0x0000000002233800 JavaThread "elasticsearch[Black Marvel][generic][T#2]"
daemon [_thread_blocked, id=10415,
stack(0x00007fc36c138000,0x00007fc36c179000)]
0x00007fc36868e800 JavaThread "elasticsearch[Black
Marvel][discovery#multicast#receiver][T#1]" daemon [_thread_in_native,
id=10413, stack(0x00007fc36c1ba000,0x00007fc36c1fb000)]
0x00007fc368675000 JavaThread "elasticsearch[Black
Marvel][transport_server_boss][T#1]{New I/O server boss #6}" daemon
[_thread_in_native, id=10412, stack(0x00007fc36c1fb000,0x00007fc36c23c000)]
0x00007fc368640800 JavaThread "elasticsearch[Black
Marvel][transport_server_worker][T#2]{New I/O worker #5}" daemon
[_thread_in_native, id=10411, stack(0x00007fc36c23c000,0x00007fc36c27d000)]
0x00007fc368640000 JavaThread "elasticsearch[Black
Marvel][transport_server_worker][T#1]{New I/O worker #4}" daemon
[_thread_in_native, id=10410, stack(0x00007fc36c27d000,0x00007fc36c2be000)]
0x00007fc36863b000 JavaThread "elasticsearch[Black
Marvel][transport_client_boss][T#1]{New I/O boss #3}" daemon
[_thread_in_native, id=10409, stack(0x00007fc36c2be000,0x00007fc36c2ff000)]
0x00007fc3685f2000 JavaThread "elasticsearch[Black
Marvel][transport_client_worker][T#2]{New I/O worker #2}" daemon
[_thread_in_native, id=10408, stack(0x00007fc36c2ff000,0x00007fc36c340000)]
0x00007fc368600800 JavaThread "elasticsearch[Black
Marvel][transport_client_worker][T#1]{New I/O worker #1}" daemon
[_thread_in_native, id=10407, stack(0x00007fc36c340000,0x00007fc36c381000)]
0x00007fc3685f7800 JavaThread "elasticsearch[Black Marvel][[ttl_expire]]"
daemon [_thread_blocked, id=10406,
stack(0x00007fc36c381000,0x00007fc36c3c2000)]
0x00007fc3685a5800 JavaThread "elasticsearch[Black
Marvel][scheduler][T#1]" daemon [_thread_blocked, id=10405,
stack(0x00007fc36c3c2000,0x00007fc36c403000)]
0x00007fc36840e000 JavaThread "elasticsearch[Black Marvel][[timer]]"
daemon [_thread_blocked, id=10404,
stack(0x00007fc36c403000,0x00007fc36c444000)]
0x0000000001f18800 JavaThread "Service Thread" daemon [_thread_blocked,
id=10402, stack(0x00007fc36d0c2000,0x00007fc36d103000)]
0x0000000001f0d000 JavaThread "C2 CompilerThread1" daemon
[_thread_blocked, id=10401, stack(0x00007fc36d103000,0x00007fc36d204000)]
=>0x0000000001f0a800 JavaThread "C2 CompilerThread0" daemon [_thread_in_vm,
id=10400, stack(0x00007fc36d204000,0x00007fc36d305000)]
0x0000000001f08800 JavaThread "Signal Dispatcher" daemon
[_thread_blocked, id=10399, stack(0x00007fc36d305000,0x00007fc36d346000)]
0x0000000001f06800 JavaThread "Surrogate Locker Thread (Concurrent GC)"
daemon [_thread_blocked, id=10398,
stack(0x00007fc376e73000,0x00007fc376eb4000)]
0x0000000001eb7800 JavaThread "Finalizer" daemon [_thread_blocked,
id=10397, stack(0x00007fc36d346000,0x00007fc36d387000)]
0x0000000001eb5000 JavaThread "Reference Handler" daemon
[_thread_blocked, id=10396, stack(0x00007fc36d387000,0x00007fc36d3c8000)]
Other Threads:
0x0000000001ead800 VMThread [stack:
0x00007fc36d3c8000,0x00007fc36d4c9000] [id=10395]
0x0000000001f23800 WatcherThread [stack:
0x00007fc36cfc1000,0x00007fc36d0c2000] [id=10403]
VM state:not at safepoint (normal execution)
VM Mutex/Monitor currently owned by a thread: ([mutex/lock_event])
[0x0000000001df59f0] CodeCache_lock - owner thread: 0x0000000001f0a800
Heap
def new generation total 76672K, used 38835K [0x00000000bae00000,
0x00000000c0130000, 0x00000000c0130000)
eden space 68160K, 44% used [0x00000000bae00000, 0x00000000bcb9cc98,
0x00000000bf090000)
from space 8512K, 100% used [0x00000000bf090000, 0x00000000bf8e0000,
0x00000000bf8e0000)
to space 8512K, 0% used [0x00000000bf8e0000, 0x00000000bf8e0000,
0x00000000c0130000)
concurrent mark-sweep generation total 176960K, used 12831K
[0x00000000c0130000, 0x00000000cae00000, 0x00000000fae00000)
concurrent-mark-sweep perm gen total 28736K, used 28714K
[0x00000000fae00000, 0x00000000fca10000, 0x0000000100000000)
Card table byte_map: [0x00007fc3711f3000,0x00007fc37141d000] byte_map_base:
0x00007fc370c1c000
Polling page: 0x00007fc377031000
Code Cache [0x00007fc37141d000, 0x00007fc37168d000, 0x00007fc37441d000)
total_blobs=671 nmethods=287 adapters=336 free_code_cache=48179Kb
largest_free_block=49272512
Compilation events (10 events):
Event: 36.001 Thread 0x0000000001f0d000 nmethod 298 0x00007fc3714d5d50 code
[0x00007fc3714d5e80, 0x00007fc3714d5ef8]
Event: 36.019 Thread 0x0000000001f0d000 299
java.util.Arrays::fill (21 bytes)
Event: 36.020 Thread 0x0000000001f0d000 nmethod 299 0x00007fc3714d5ad0 code
[0x00007fc3714d5c00, 0x00007fc3714d5c98]
Event: 36.037 Thread 0x0000000001f0d000 300
org.apache.lucene.util.PriorityQueue::downHeap (138 bytes)
Event: 36.041 Thread 0x0000000001f0d000 nmethod 300 0x00007fc3714e3a50 code
[0x00007fc3714e3be0, 0x00007fc3714e3f98]
Event: 68.193 Thread 0x0000000001f0d000 301
java.util.concurrent.atomic.AtomicInteger::get (5 bytes)
Event: 68.193 Thread 0x0000000001f0d000 nmethod 301 0x00007fc3714e3850 code
[0x00007fc3714e3980, 0x00007fc3714e39d8]
Event: 97.398 Thread 0x0000000001f0a800 302
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask::compareTo
(9 bytes)
Event: 97.398 Thread 0x0000000001f0d000 303
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask::compareTo
(101 bytes)
Event: 97.399 Thread 0x0000000001f0d000 nmethod 303 0x00007fc3714e4350 code
[0x00007fc3714e44a0, 0x00007fc3714e4578]
GC Heap History (4 events):
Event: 2.811 GC heap before
{Heap before GC invocations=0 (full 0):
def new generation total 76672K, used 68160K [0x00000000bae00000,
0x00000000c0130000, 0x00000000c0130000)
eden space 68160K, 100% used [0x00000000bae00000, 0x00000000bf090000,
0x00000000bf090000)
from space 8512K, 0% used [0x00000000bf090000, 0x00000000bf090000,
0x00000000bf8e0000)
to space 8512K, 0% used [0x00000000bf8e0000, 0x00000000bf8e0000,
0x00000000c0130000)
concurrent mark-sweep generation total 176960K, used 0K
[0x00000000c0130000, 0x00000000cae00000, 0x00000000fae00000)
concurrent-mark-sweep perm gen total 21248K, used 19010K
[0x00000000fae00000, 0x00000000fc2c0000, 0x0000000100000000)
Event: 2.845 GC heap after
Heap after GC invocations=1 (full 0):
def new generation total 76672K, used 8512K [0x00000000bae00000,
0x00000000c0130000, 0x00000000c0130000)
eden space 68160K, 0% used [0x00000000bae00000, 0x00000000bae00000,
0x00000000bf090000)
from space 8512K, 100% used [0x00000000bf8e0000, 0x00000000c0130000,
0x00000000c0130000)
to space 8512K, 0% used [0x00000000bf090000, 0x00000000bf090000,
0x00000000bf8e0000)
concurrent mark-sweep generation total 176960K, used 7376K
[0x00000000c0130000, 0x00000000cae00000, 0x00000000fae00000)
concurrent-mark-sweep perm gen total 21248K, used 19010K
[0x00000000fae00000, 0x00000000fc2c0000, 0x0000000100000000)
}
Event: 7.194 GC heap before
{Heap before GC invocations=1 (full 0):
def new generation total 76672K, used 76672K [0x00000000bae00000,
0x00000000c0130000, 0x00000000c0130000)
eden space 68160K, 100% used [0x00000000bae00000, 0x00000000bf090000,
0x00000000bf090000)
from space 8512K, 100% used [0x00000000bf8e0000, 0x00000000c0130000,
0x00000000c0130000)
to space 8512K, 0% used [0x00000000bf090000, 0x00000000bf090000,
0x00000000bf8e0000)
concurrent mark-sweep generation total 176960K, used 7376K
[0x00000000c0130000, 0x00000000cae00000, 0x00000000fae00000)
concurrent-mark-sweep perm gen total 27008K, used 26913K
[0x00000000fae00000, 0x00000000fc860000, 0x0000000100000000)
Event: 7.237 GC heap after
Heap after GC invocations=2 (full 0):
def new generation total 76672K, used 8512K [0x00000000bae00000,
0x00000000c0130000, 0x00000000c0130000)
eden space 68160K, 0% used [0x00000000bae00000, 0x00000000bae00000,
0x00000000bf090000)
from space 8512K, 100% used [0x00000000bf090000, 0x00000000bf8e0000,
0x00000000bf8e0000)
to space 8512K, 0% used [0x00000000bf8e0000, 0x00000000bf8e0000,
0x00000000c0130000)
concurrent mark-sweep generation total 176960K, used 12831K
[0x00000000c0130000, 0x00000000cae00000, 0x00000000fae00000)
concurrent-mark-sweep perm gen total 27008K, used 26913K
[0x00000000fae00000, 0x00000000fc860000, 0x0000000100000000)
}
Deoptimization events (10 events):
Event: 6.537 Thread 0x0000000002234800 Uncommon trap -34 fr.pc
0x00007fc3714e9550
Event: 6.654 Thread 0x0000000002234800 Uncommon trap -34 fr.pc
0x00007fc3714d4de0
Event: 6.654 Thread 0x0000000002234800 Uncommon trap -34 fr.pc
0x00007fc3714d4de0
Event: 6.654 Thread 0x0000000002234800 Uncommon trap -34 fr.pc
0x00007fc3714d4de0
Event: 6.654 Thread 0x0000000002234800 Uncommon trap -34 fr.pc
0x00007fc3714d4de0
Event: 6.654 Thread 0x0000000002234800 Uncommon trap -34 fr.pc
0x00007fc3714def74
Event: 6.654 Thread 0x0000000002234800 Uncommon trap -34 fr.pc
0x00007fc3714def74
Event: 6.654 Thread 0x0000000002234800 Uncommon trap -34 fr.pc
0x00007fc3714def74
Event: 6.655 Thread 0x0000000002234800 Uncommon trap -34 fr.pc
0x00007fc3714def74
Event: 6.655 Thread 0x0000000002234800 Uncommon trap -34 fr.pc
0x00007fc3714d894c
Internal exceptions (10 events):
Event: 35.841 Thread 0x00000000022c4800 Threw 0x00000000bc924370 at
/HUDSON/workspace/jdk7u25-2-build-linux-amd64-product/jdk7u25/hotspot/src/share/vm/prims/jvm.cpp:1235
Event: 35.841 Thread 0x00000000022c4800 Threw 0x00000000bc9278c0 at
/HUDSON/workspace/jdk7u25-2-build-linux-amd64-product/jdk7u25/hotspot/src/share/vm/prims/jvm.cpp:1235
Event: 35.842 Thread 0x00000000022c4800 Threw 0x00000000bc92e5f8 at
/HUDSON/workspace/jdk7u25-2-build-linux-amd64-product/jdk7u25/hotspot/src/share/vm/prims/jvm.cpp:1235
Event: 35.843 Thread 0x00000000022c4800 Threw 0x00000000bc931138 at
/HUDSON/workspace/jdk7u25-2-build-linux-amd64-product/jdk7u25/hotspot/src/share/vm/prims/jvm.cpp:1235
Event: 35.843 Thread 0x00000000022c4800 Threw 0x00000000bc935090 at
/HUDSON/workspace/jdk7u25-2-build-linux-amd64-product/jdk7u25/hotspot/src/share/vm/prims/jvm.cpp:1235
Event: 35.844 Thread 0x00000000022c4800 Threw 0x00000000bc93f2b8 at
/HUDSON/workspace/jdk7u25-2-build-linux-amd64-product/jdk7u25/hotspot/src/share/vm/prims/jvm.cpp:1235
Event: 35.846 Thread 0x00007fc368242000 Threw 0x00000000bb8317b0 at
/HUDSON/workspace/jdk7u25-2-build-linux-amd64-product/jdk7u25/hotspot/src/share/vm/prims/jvm.cpp:1235
Event: 35.847 Thread 0x00007fc368242000 Threw 0x00000000bb837450 at
/HUDSON/workspace/jdk7u25-2-build-linux-amd64-product/jdk7u25/hotspot/src/share/vm/prims/jvm.cpp:1235
Event: 35.847 Thread 0x00007fc368242000 Threw 0x00000000bb83b5f8 at
/HUDSON/workspace/jdk7u25-2-build-linux-amd64-product/jdk7u25/hotspot/src/share/vm/prims/jvm.cpp:1235
Event: 64.973 Thread 0x00007fc36868e800 Threw 0x00000000bcaedfe8 at
/HUDSON/workspace/jdk7u25-2-build-linux-amd64-product/jdk7u25/hotspot/src/share/vm/prims/jni.cpp:742
Events (10 events):
Event: 36.046 Executing VM operation: RevokeBias
Event: 36.046 Executing VM operation: RevokeBias done
Event: 37.267 Thread 0x00007fc368692000 Thread exited: 0x00007fc368692000
Event: 52.893 Executing VM operation: RevokeBias
Event: 52.893 Executing VM operation: RevokeBias done
Event: 68.193 Thread 0x0000000001f0d000 flushing nmethod 0x00007fc37149c9d0
Event: 68.193 Thread 0x0000000001f0d000 flushing nmethod 0x00007fc37149df50
Event: 68.193 Thread 0x0000000001f0d000 flushing nmethod 0x00007fc37149fb90
Event: 97.399 Thread 0x0000000001f0d000 flushing nmethod 0x00007fc3714a0350
Event: 97.399 Thread 0x0000000001f0d000 flushing nmethod 0x00007fc3714a4b90

Dynamic libraries:
00400000-00401000 r-xp 00000000 fd:02 1794093
/usr/lib/jvm/java-7-oracle/bin/java
00600000-00601000 rw-p 00000000 fd:02 1794093
/usr/lib/jvm/java-7-oracle/bin/java
01def000-0276f000 rw-p 00000000 00:00 0
[heap]
bae00000-cae00000 rw-p 00000000 00:00 0
cae00000-fae00000 rw-p 00000000 00:00 0
fae00000-fca10000 rw-p 00000000 00:00 0
fca10000-100000000 rw-p 00000000 00:00 0
7fc367c7d000-7fc367db7000 rw-p 00000000 00:00 0
7fc367db7000-7fc367dba000 ---p 00000000 00:00 0
7fc367dba000-7fc367df8000 rw-p 00000000 00:00 0
7fc367df8000-7fc367dfb000 ---p 00000000 00:00 0
7fc367dfb000-7fc367e39000 rw-p 00000000 00:00 0
7fc367e39000-7fc367e3c000 ---p 00000000 00:00 0
7fc367e3c000-7fc367e7a000 rw-p 00000000 00:00 0
7fc367e7a000-7fc367e7d000 ---p 00000000 00:00 0
7fc367e7d000-7fc367ebb000 rw-p 00000000 00:00 0
7fc367ebb000-7fc367ebe000 ---p 00000000 00:00 0
7fc367ebe000-7fc367efc000 rw-p 00000000 00:00 0
7fc367efc000-7fc367eff000 ---p 00000000 00:00 0
7fc367eff000-7fc367f3d000 rw-p 00000000 00:00 0
7fc367f3d000-7fc367f40000 ---p 00000000 00:00 0
7fc367f40000-7fc367f7e000 rw-p 00000000 00:00 0
7fc367f7e000-7fc367f81000 ---p 00000000 00:00 0
7fc367f81000-7fc367fbf000 rw-p 00000000 00:00 0
7fc367fbf000-7fc367fc2000 ---p 00000000 00:00 0
7fc367fc2000-7fc368000000 rw-p 00000000 00:00 0
7fc368000000-7fc36871a000 rw-p 00000000 00:00 0
7fc36871a000-7fc36c000000 ---p 00000000 00:00 0
7fc36c034000-7fc36c037000 ---p 00000000 00:00 0
7fc36c037000-7fc36c075000 rw-p 00000000 00:00 0
7fc36c075000-7fc36c078000 ---p 00000000 00:00 0
7fc36c078000-7fc36c0b6000 rw-p 00000000 00:00 0
7fc36c0b6000-7fc36c0b9000 ---p 00000000 00:00 0
7fc36c0b9000-7fc36c0f7000 rw-p 00000000 00:00 0
7fc36c0f7000-7fc36c0fa000 ---p 00000000 00:00 0
7fc36c0fa000-7fc36c138000 rw-p 00000000 00:00 0
7fc36c138000-7fc36c13b000 ---p 00000000 00:00 0
7fc36c13b000-7fc36c179000 rw-p 00000000 00:00 0
7fc36c179000-7fc36c17c000 ---p 00000000 00:00 0
7fc36c17c000-7fc36c1ba000 rw-p 00000000 00:00 0
7fc36c1ba000-7fc36c1bd000 ---p 00000000 00:00 0
7fc36c1bd000-7fc36c1fb000 rw-p 00000000 00:00 0
7fc36c1fb000-7fc36c1fe000 ---p 00000000 00:00 0
7fc36c1fe000-7fc36c23c000 rw-p 00000000 00:00 0
7fc36c23c000-7fc36c23f000 ---p 00000000 00:00 0
7fc36c23f000-7fc36c27d000 rw-p 00000000 00:00 0
7fc36c27d000-7fc36c280000 ---p 00000000 00:00 0
7fc36c280000-7fc36c2be000 rw-p 00000000 00:00 0
7fc36c2be000-7fc36c2c1000 ---p 00000000 00:00 0
7fc36c2c1000-7fc36c2ff000 rw-p 00000000 00:00 0
7fc36c2ff000-7fc36c302000 ---p 00000000 00:00 0
7fc36c302000-7fc36c340000 rw-p 00000000 00:00 0
7fc36c340000-7fc36c343000 ---p 00000000 00:00 0
7fc36c343000-7fc36c381000 rw-p 00000000 00:00 0
7fc36c381000-7fc36c384000 ---p 00000000 00:00 0
7fc36c384000-7fc36c3c2000 rw-p 00000000 00:00 0
7fc36c3c2000-7fc36c3c5000 ---p 00000000 00:00 0
7fc36c3c5000-7fc36c403000 rw-p 00000000 00:00 0
7fc36c403000-7fc36c406000 ---p 00000000 00:00 0
7fc36c406000-7fc36c444000 rw-p 00000000 00:00 0
7fc36c444000-7fc36c473000 r-xp 00000000 fd:02 1532116
/usr/share/elasticsearch/lib/sigar/libsigar-amd64-linux.so
7fc36c473000-7fc36c572000 ---p 0002f000 fd:02 1532116
/usr/share/elasticsearch/lib/sigar/libsigar-amd64-linux.so
7fc36c572000-7fc36c578000 rw-p 0002e000 fd:02 1532116
/usr/share/elasticsearch/lib/sigar/libsigar-amd64-linux.so
7fc36c578000-7fc36c57b000 rw-p 00000000 00:00 0
7fc36c57b000-7fc36c664000 r-xp 00000000 fd:02 6901093
/tmp/snappy-1.0.4.1-libsnappyjava.so
7fc36c664000-7fc36c864000 ---p 000e9000 fd:02 6901093
/tmp/snappy-1.0.4.1-libsnappyjava.so
7fc36c864000-7fc36c86d000 rw-p 000e9000 fd:02 6901093
/tmp/snappy-1.0.4.1-libsnappyjava.so
7fc36c86d000-7fc36c881000 rw-p 00000000 00:00 0
7fc36c881000-7fc36c885000 r--s 0007d000 fd:02 1328181
/usr/lib/jvm/java-7-oracle/lib/jsse.jar
7fc36c885000-7fc36c895000 r-xp 00000000 fd:02 1441857
/usr/lib/jvm/java-7-oracle/lib/amd64/libnio.so
7fc36c895000-7fc36ca95000 ---p 00010000 fd:02 1441857
/usr/lib/jvm/java-7-oracle/lib/amd64/libnio.so
7fc36ca95000-7fc36ca96000 rw-p 00010000 fd:02 1441857
/usr/lib/jvm/java-7-oracle/lib/amd64/libnio.so
7fc36ca96000-7fc36caab000 r-xp 00000000 fd:02 1441817
/usr/lib/jvm/java-7-oracle/lib/amd64/libnet.so
7fc36caab000-7fc36ccab000 ---p 00015000 fd:02 1441817
/usr/lib/jvm/java-7-oracle/lib/amd64/libnet.so
7fc36ccab000-7fc36ccac000 rw-p 00015000 fd:02 1441817
/usr/lib/jvm/java-7-oracle/lib/amd64/libnet.so
7fc36ccac000-7fc36ccb4000 r-xp 00000000 fd:02 1441819
/usr/lib/jvm/java-7-oracle/lib/amd64/libmanagement.so
7fc36ccb4000-7fc36ceb3000 ---p 00008000 fd:02 1441819
/usr/lib/jvm/java-7-oracle/lib/amd64/libmanagement.so
7fc36ceb3000-7fc36ceb4000 rw-p 00007000 fd:02 1441819
/usr/lib/jvm/java-7-oracle/lib/amd64/libmanagement.so
7fc36ceb4000-7fc36cebb000 r--s 00062000 fd:02 9805827
/usr/share/elasticsearch/lib/sigar/sigar-1.6.4.jar
7fc36cebb000-7fc36cec6000 r--s 00114000 fd:02 9797636
/usr/share/elasticsearch/lib/lucene-analyzers-3.6.2.jar
7fc36cec6000-7fc36cedf000 r--s 0015f000 fd:02 9797637
/usr/share/elasticsearch/lib/lucene-core-3.6.2.jar
7fc36cedf000-7fc36cee3000 r--s 000d0000 fd:02 9797633
/usr/share/elasticsearch/lib/jna-3.3.0.jar
7fc36cee3000-7fc36cef4000 r--s 000ac000 fd:02 9797634
/usr/share/elasticsearch/lib/jts-1.12.jar
7fc36cef4000-7fc36cfc1000 r--s 0095d000 fd:02 1532115
/usr/share/elasticsearch/lib/elasticsearch-0.20.6.jar
7fc36cfc1000-7fc36cfc2000 ---p 00000000 00:00 0
7fc36cfc2000-7fc36d0c2000 rw-p 00000000 00:00 0
7fc36d0c2000-7fc36d0c5000 ---p 00000000 00:00 0
7fc36d0c5000-7fc36d103000 rw-p 00000000 00:00 0
7fc36d103000-7fc36d106000 ---p 00000000 00:00 0
7fc36d106000-7fc36d204000 rw-p 00000000 00:00 0
7fc36d204000-7fc36d207000 ---p 00000000 00:00 0
7fc36d207000-7fc36d305000 rw-p 00000000 00:00 0
7fc36d305000-7fc36d308000 ---p 00000000 00:00 0
7fc36d308000-7fc36d346000 rw-p 00000000 00:00 0
7fc36d346000-7fc36d349000 ---p 00000000 00:00 0
7fc36d349000-7fc36d387000 rw-p 00000000 00:00 0
7fc36d387000-7fc36d38a000 ---p 00000000 00:00 0
7fc36d38a000-7fc36d3c8000 rw-p 00000000 00:00 0
7fc36d3c8000-7fc36d3c9000 ---p 00000000 00:00 0
7fc36d3c9000-7fc36d539000 rw-p 00000000 00:00 0
7fc36d539000-7fc36d6f7000 r--s 031d8000 fd:02 1328167
/usr/lib/jvm/java-7-oracle/lib/rt.jar
7fc36d6f7000-7fc36d71f000 rw-p 00000000 00:00 0
7fc36d71f000-7fc36d720000 ---p 00000000 00:00 0
7fc36d720000-7fc371073000 rw-p 00000000 00:00 0
7fc371073000-7fc3711f3000 rw-p 00000000 00:00 0
7fc3711f3000-7fc371273000 rw-p 00000000 00:00 0
7fc371273000-7fc3713f3000 rw-p 00000000 00:00 0
7fc3713f3000-7fc371402000 rw-p 00000000 00:00 0
7fc371402000-7fc37141c000 rw-p 00000000 00:00 0
7fc37141c000-7fc37141d000 rw-p 00000000 00:00 0
7fc37141d000-7fc37168d000 rwxp 00000000 00:00 0
7fc37168d000-7fc37441d000 rw-p 00000000 00:00 0
7fc37441d000-7fc374438000 r-xp 00000000 fd:02 1441835
/usr/lib/jvm/java-7-oracle/lib/amd64/libzip.so
7fc374438000-7fc374637000 ---p 0001b000 fd:02 1441835
/usr/lib/jvm/java-7-oracle/lib/amd64/libzip.so
7fc374637000-7fc374638000 rw-p 0001a000 fd:02 1441835
/usr/lib/jvm/java-7-oracle/lib/amd64/libzip.so
7fc374638000-7fc374644000 r-xp 00000000 fd:02 6905598
/lib/libnss_files-2.11.1.so
7fc374644000-7fc374843000 ---p 0000c000 fd:02 6905598
/lib/libnss_files-2.11.1.so
7fc374843000-7fc374844000 r--p 0000b000 fd:02 6905598
/lib/libnss_files-2.11.1.so
7fc374844000-7fc374845000 rw-p 0000c000 fd:02 6905598
/lib/libnss_files-2.11.1.so
7fc374845000-7fc37484f000 r-xp 00000000 fd:02 6905601
/lib/libnss_nis-2.11.1.so
7fc37484f000-7fc374a4e000 ---p 0000a000 fd:02 6905601
/lib/libnss_nis-2.11.1.so
7fc374a4e000-7fc374a4f000 r--p 00009000 fd:02 6905601
/lib/libnss_nis-2.11.1.so
7fc374a4f000-7fc374a50000 rw-p 0000a000 fd:02 6905601
/lib/libnss_nis-2.11.1.so
7fc374a50000-7fc374a67000 r-xp 00000000 fd:02 6905594
/lib/libnsl-2.11.1.so
7fc374a67000-7fc374c66000 ---p 00017000 fd:02 6905594
/lib/libnsl-2.11.1.so
7fc374c66000-7fc374c67000 r--p 00016000 fd:02 6905594
/lib/libnsl-2.11.1.so
7fc374c67000-7fc374c68000 rw-p 00017000 fd:02 6905594
/lib/libnsl-2.11.1.so
7fc374c68000-7fc374c6a000 rw-p 00000000 00:00 0
7fc374c6a000-7fc374c72000 r-xp 00000000 fd:02 6905586
/lib/libnss_compat-2.11.1.so
7fc374c72000-7fc374e71000 ---p 00008000 fd:02 6905586
/lib/libnss_compat-2.11.1.so
7fc374e71000-7fc374e72000 r--p 00007000 fd:02 6905586
/lib/libnss_compat-2.11.1.so
7fc374e72000-7fc374e73000 rw-p 00008000 fd:02 6905586
/lib/libnss_compat-2.11.1.so
7fc374e73000-7fc374e9c000 r-xp 00000000 fd:02 1441832
/usr/lib/jvm/java-7-oracle/lib/amd64/libjava.so
7fc374e9c000-7fc37509c000 ---p 00029000 fd:02 1441832
/usr/lib/jvm/java-7-oracle/lib/amd64/libjava.so
7fc37509c000-7fc37509e000 rw-p 00029000 fd:02 1441832
/usr/lib/jvm/java-7-oracle/lib/amd64/libjava.so
7fc37509e000-7fc3750ab000 r-xp 00000000 fd:02 1441822
/usr/lib/jvm/java-7-oracle/lib/amd64/libverify.so
7fc3750ab000-7fc3752aa000 ---p 0000d000 fd:02 1441822
/usr/lib/jvm/java-7-oracle/lib/amd64/libverify.so
7fc3752aa000-7fc3752ac000 rw-p 0000c000 fd:02 1441822
/usr/lib/jvm/java-7-oracle/lib/amd64/libverify.so
7fc3752ac000-7fc3752b3000 r-xp 00000000 fd:02 6905587
/lib/librt-2.11.1.so
7fc3752b3000-7fc3754b2000 ---p 00007000 fd:02 6905587
/lib/librt-2.11.1.so
7fc3754b2000-7fc3754b3000 r--p 00006000 fd:02 6905587
/lib/librt-2.11.1.so
7fc3754b3000-7fc3754b4000 rw-p 00007000 fd:02 6905587
/lib/librt-2.11.1.so
7fc3754b4000-7fc375536000 r-xp 00000000 fd:02 6905582
/lib/libm-2.11.1.so
7fc375536000-7fc375735000 ---p 00082000 fd:02 6905582
/lib/libm-2.11.1.so
7fc375735000-7fc375736000 r--p 00081000 fd:02 6905582
/lib/libm-2.11.1.so
7fc375736000-7fc375737000 rw-p 00082000 fd:02 6905582
/lib/libm-2.11.1.so
7fc375737000-7fc376178000 r-xp 00000000 fd:02 1794080
/usr/lib/jvm/java-7-oracle/lib/amd64/server/libjvm.so
7fc376178000-7fc376378000 ---p 00a41000 fd:02 1794080
/usr/lib/jvm/java-7-oracle/lib/amd64/server/libjvm.so
7fc376378000-7fc37641b000 rw-p 00a41000 fd:02 1794080
/usr/lib/jvm/java-7-oracle/lib/amd64/server/libjvm.so
7fc37641b000-7fc376457000 rw-p 00000000 00:00 0
7fc376457000-7fc3765d4000 r-xp 00000000 fd:02 6905599
/lib/libc-2.11.1.so
7fc3765d4000-7fc3767d3000 ---p 0017d000 fd:02 6905599
/lib/libc-2.11.1.so
7fc3767d3000-7fc3767d7000 r--p 0017c000 fd:02 6905599
/lib/libc-2.11.1.so
7fc3767d7000-7fc3767d8000 rw-p 00180000 fd:02 6905599
/lib/libc-2.11.1.so
7fc3767d8000-7fc3767dd000 rw-p 00000000 00:00 0
7fc3767dd000-7fc3767df000 r-xp 00000000 fd:02 6905597
/lib/libdl-2.11.1.so
7fc3767df000-7fc3769df000 ---p 00002000 fd:02 6905597
/lib/libdl-2.11.1.so
7fc3769df000-7fc3769e0000 r--p 00002000 fd:02 6905597
/lib/libdl-2.11.1.so
7fc3769e0000-7fc3769e1000 rw-p 00003000 fd:02 6905597
/lib/libdl-2.11.1.so
7fc3769e1000-7fc3769f7000 r-xp 00000000 fd:02 1794083
/usr/lib/jvm/java-7-oracle/lib/amd64/jli/libjli.so
7fc3769f7000-7fc376bf7000 ---p 00016000 fd:02 1794083
/usr/lib/jvm/java-7-oracle/lib/amd64/jli/libjli.so
7fc376bf7000-7fc376bf8000 rw-p 00016000 fd:02 1794083
/usr/lib/jvm/java-7-oracle/lib/amd64/jli/libjli.so
7fc376bf8000-7fc376c10000 r-xp 00000000 fd:02 6905593
/lib/libpthread-2.11.1.so
7fc376c10000-7fc376e0f000 ---p 00018000 fd:02 6905593
/lib/libpthread-2.11.1.so
7fc376e0f000-7fc376e10000 r--p 00017000 fd:02 6905593
/lib/libpthread-2.11.1.so
7fc376e10000-7fc376e11000 rw-p 00018000 fd:02 6905593
/lib/libpthread-2.11.1.so
7fc376e11000-7fc376e15000 rw-p 00000000 00:00 0
7fc376e15000-7fc376e35000 r-xp 00000000 fd:02 6905590
/lib/ld-2.11.1.so
7fc376e37000-7fc376e39000 r--s 00014000 fd:02 9797638
/usr/share/elasticsearch/lib/lucene-highlighter-3.6.2.jar
7fc376e39000-7fc376e3b000 r--s 000f2000 fd:02 9797641
/usr/share/elasticsearch/lib/snappy-java-1.0.4.1.jar
7fc376e3b000-7fc376e3d000 r--s 0000a000 fd:02 9797640
/usr/share/elasticsearch/lib/lucene-queries-3.6.2.jar
7fc376e3d000-7fc376e46000 r--s 0006f000 fd:02 9797635
/usr/share/elasticsearch/lib/log4j-1.2.17.jar
7fc376e46000-7fc376e73000 rw-p 00000000 00:00 0
7fc376e73000-7fc376e76000 ---p 00000000 00:00 0
7fc376e76000-7fc376f03000 rw-p 00000000 00:00 0
7fc376f03000-7fc376f1e000 rw-p 00000000 00:00 0
7fc376f1e000-7fc376f28000 rw-p 00000000 00:00 0
7fc376f28000-7fc376fde000 rw-p 00000000 00:00 0
7fc376fde000-7fc376fe6000 rw-s 00000000 fd:02 829991
/tmp/hsperfdata_root/10387
7fc376fe6000-7fc376fe9000 ---p 00000000 00:00 0
7fc376fe9000-7fc37702c000 rw-p 00000000 00:00 0
7fc37702c000-7fc37702e000 r--s 00006000 fd:02 9797639
/usr/share/elasticsearch/lib/lucene-memory-3.6.2.jar
7fc37702e000-7fc377030000 r--s 0000f000 fd:02 9797642
/usr/share/elasticsearch/lib/spatial4j-0.3.jar
7fc377030000-7fc377031000 rw-p 00000000 00:00 0
7fc377031000-7fc377032000 r--p 00000000 00:00 0
7fc377032000-7fc377034000 rw-p 00000000 00:00 0
7fc377034000-7fc377035000 r--p 0001f000 fd:02 6905590
/lib/ld-2.11.1.so
7fc377035000-7fc377036000 rw-p 00020000 fd:02 6905590
/lib/ld-2.11.1.so
7fc377036000-7fc377037000 rw-p 00000000 00:00 0
7fff219c1000-7fff219d6000 rw-p 00000000 00:00 0
[stack]
7fff219fe000-7fff21a00000 r-xp 00000000 00:00 0
[vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0
[vsyscall]
VM Arguments:
jvm_args: -Xms256m -Xmx1g -Xss256k -Djava.awt.headless=true
-XX:+UseParNewGC -XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly
-XX:+HeapDumpOnOutOfMemoryError -Delasticsearch
-Des.pidfile=/var/run/elasticsearch.pid -Des.foreground=yes
-Des.path.home=/usr/share/elasticsearch
-Des.default.config=/etc/elasticsearch/elasticsearch.yml
-Des.default.path.home=/usr/share/elasticsearch
-Des.default.path.logs=/var/log/elasticsearch
-Des.default.path.data=/var/lib/elasticsearch
-Des.default.path.work=/tmp/elasticsearch
-Des.default.path.conf=/etc/elasticsearch
java_command: org.elasticsearch.bootstrap.Elasticsearch
Launcher Type: SUN_STANDARD
Environment Variables:
JAVA_HOME=/usr/lib/jvm/java-7-oracle
PATH=/bin:/usr/bin:/sbin:/usr/sbin
SHELL=/bin/bash
Signal Handlers:
SIGSEGV: [libjvm.so+0x8a26a0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGBUS: [libjvm.so+0x8a26a0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGFPE: [libjvm.so+0x73f3d0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGPIPE: [libjvm.so+0x73f3d0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGXFSZ: [libjvm.so+0x73f3d0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGILL: [libjvm.so+0x73f3d0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGUSR1: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGUSR2: [libjvm.so+0x73ed20], sa_mask[0]=0x00000000, sa_flags=0x10000004
SIGHUP: [libjvm.so+0x7410b0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGINT: [libjvm.so+0x7410b0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGTERM: [libjvm.so+0x7410b0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGQUIT: [libjvm.so+0x7410b0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004

--------------- S Y S T E M ---------------
OS:squeeze/sid
uname:Linux 2.6.32-6-pve #1 SMP Mon Jan 23 08:27:52 CET 2012 x86_64
libc:glibc 2.11.1 NPTL 2.11.1
rlimit: STACK 8192k, CORE 0k, NPROC infinity, NOFILE 1024, AS infinity
load average:0.08 0.05 0.01
/proc/meminfo:
MemTotal: 4194304 kB
MemFree: 2021064 kB
Cached: 197304 kB
Active: 732900 kB
Inactive: 35624 kB
Active(anon): 571216 kB
Inactive(anon): 4 kB
Active(file): 161684 kB
Inactive(file): 35620 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 48 kB
AnonPages: 571220 kB
Shmem: 2560 kB
Slab: 50220 kB
SReclaimable: 38220 kB
SUnreclaim: 12000 kB

CPU:total 1 (4 cores per cpu, 2 threads per core) family 6 model 44
stepping 2, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2,
popcnt, ht, tsc, tscinvbit, tscinv
/proc/cpuinfo:
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 44
model name : Intel(R) Xeon(R) CPU E5620 @ 2.40GHz
stepping : 2
cpu MHz : 2394.012
cache size : 12288 KB
physical id : 1
siblings : 8
core id : 0
cpu cores : 4
apicid : 32
initial apicid : 32
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov
pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology
nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2
ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt aes lahf_lm ida arat
tpr_shadow vnmi flexpriority ept vpid
bogomips : 4788.02
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:

Memory: 4k page, physical 4194304k(2021064k free), swap 0k(0k free)
vm_info: Java HotSpot(TM) 64-Bit Server VM (23.25-b01) for linux-amd64 JRE
(1.7.0_25-b15), built on Jun 5 2013 20:48:36 by "java_re" with gcc 4.3.0
20080428 (Red Hat 4.3.0-8)
time: Thu Jul 4 10:26:58 2013
elapsed time: 97 seconds

Any idea on what's going on ?

I played with max_open_file too without any success or other memory related
stuff...

Many thanks,
DJP

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

jprante · July 4, 2013, 6:33pm

First of all, your full dump is from a crash with Java 7u25 and ES
0.20.6 (not 0.90.2)

Your crash is due to the HotSpot Compiler in the JVM, not ES.

Probably related to the fact you did not reserve any swap memory on your
machine, but I'm not sure.

Can you check if ES works with Java 7u21?

http://www.oracle.com/technetwork/java/javase/downloads/java-archive-downloads-javase7-521261.html

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

DJP · July 5, 2013, 9:50am

Hello Jörg,

Many thanks for your answer. You're right about the dump, my mistake...

Here's one with Java 7u21 and ES 0.90.2. Furthermore, i've checked my swap
(1024M) and raised up to 2048M. (I'm in an openvz environment...)

[thread 140189111027456 also had an error]# A fatal error has been
detected by the Java Runtime Environment:

Internal Error (assembler_x86.cpp:746), pid=14753, tid=140189122754304

Error: ShouldNotReachHere()

JRE version: 7.0_21-b11

Java VM: Java HotSpot(TM) 64-Bit Server VM (23.21-b01 mixed mode

linux-amd64 compressed oops)

Failed to write core dump. Core dumps have been disabled. To enable core

dumping, try "ulimit -c unlimited" before starting Java again

An error report file with more information is saved as:

/usr/lib/jvm/hs_err_pid14753.log

If you would like to submit a bug report, please visit:

http://bugreport.sun.com/bugreport/crash.jsp

Aborted

And the full backtrace :

A fatal error has been detected by the Java Runtime Environment:

Internal Error (assembler_x86.cpp:746), pid=14753, tid=140189122754304

Error: ShouldNotReachHere()

JRE version: 7.0_21-b11

Java VM: Java HotSpot(TM) 64-Bit Server VM (23.21-b01 mixed mode

linux-amd64 compressed oops)

Failed to write core dump. Core dumps have been disabled. To enable core

dumping, try "ulimit -c unlimited" before starting Java again

If you would like to submit a bug report, please visit:

http://bugreport.sun.com/bugreport/crash.jsp

--------------- T H R E A D ---------------
Current thread (0x000000000198b000): JavaThread "C2 CompilerThread1"
daemon [_thread_in_vm, id=14767,
stack(0x00007f8052ccf000,0x00007f8052dd0000)]
Stack: [0x00007f8052ccf000,0x00007f8052dd0000], sp=0x00007f8052dccc30,
free space=1015k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
code)
V [libjvm.so+0x8a777a] VMError::report_and_die()+0x2ea
V [libjvm.so+0x40b792] report_should_not_reach_here(char const*, int)+0x52
V [libjvm.so+0x23cf31] Assembler::locate_operand(unsigned char*,
Assembler::WhichOperand)+0xd1
V [libjvm.so+0x7c0789] Relocation::pd_set_data_value(unsigned char*,
long, bool)+0x99
V [libjvm.so+0x7166d0] nmethod::fix_oop_relocations(unsigned char*,
unsigned char*, bool)+0x120
V [libjvm.so+0x7194e3] nmethod::nmethod(methodOopDesc*, int, int, int,
CodeOffsets*, int, DebugInformationRecorder*, Dependencies*, CodeBuffer*,
int, OopMapSet*, ExceptionHandlerTable*, ImplicitExceptionTable*,
AbstractCompiler*, int)+0x2e3
V [libjvm.so+0x7198ad] nmethod::new_nmethod(methodHandle, int, int,
CodeOffsets*, int, DebugInformationRecorder*, Dependencies*, CodeBuffer*,
int, OopMapSet*, ExceptionHandlerTable*, ImplicitExceptionTable*,
AbstractCompiler*, int)+0x1fd
V [libjvm.so+0x34cb82] ciEnv::register_method(ciMethod*, int,
CodeOffsets*, int, CodeBuffer*, int, OopMapSet*, ExceptionHandlerTable*,
ImplicitExceptionTable*, AbstractCompiler*, int, bool)+0x352
V [libjvm.so+0x3bd472] Compile::Compile(ciEnv*, C2Compiler*, ciMethod*,
int, bool, bool)+0xee2
V [libjvm.so+0x32f082] C2Compiler::compile_method(ciEnv*, ciMethod*,
int)+0x142
V [libjvm.so+0x3c22e3]
CompileBroker::invoke_compiler_on_method(CompileTask*)+0x323
V [libjvm.so+0x3c344d] CompileBroker::compiler_thread_loop()+0x43d
V [libjvm.so+0x867578] JavaThread::thread_main_inner()+0xc8
V [libjvm.so+0x8676c8] JavaThread::run()+0x138
V [libjvm.so+0x749010] java_start(Thread*)+0x100

Current CompileTask:
C2: 97704 342
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask::compareTo
(9 bytes)

--------------- P R O C E S S ---------------
Java Threads: ( => current thread )
0x0000000001f3c800 JavaThread "elasticsearch[Argus][search][T#3]" daemon
[_thread_blocked, id=14792, stack(0x00007f8051d07000,0x00007f8051d48000)]
0x0000000001b87800 JavaThread "elasticsearch[Argus][search][T#2]" daemon
[_thread_blocked, id=14791, stack(0x00007f8051d48000,0x00007f8051d89000)]
0x0000000001b86800 JavaThread "elasticsearch[Argus][search][T#1]" daemon
[_thread_blocked, id=14790, stack(0x00007f8051d89000,0x00007f8051dca000)]
0x0000000002122000 JavaThread "elasticsearch[Argus][warmer][T#1]" daemon
[_thread_blocked, id=14789, stack(0x00007f8051dca000,0x00007f8051e0b000)]
0x00007f804c024800 JavaThread "DestroyJavaVM" [_thread_blocked, id=14759,
stack(0x00007f805cbba000,0x00007f805cbfb000)]
0x00007f804c023800 JavaThread "elasticsearch[keepAlive/0.90.2]"
[_thread_blocked, id=14788, stack(0x00007f8051e0b000,0x00007f8051e4c000)]
0x00007f804c2ca800 JavaThread
"elasticsearch[Argus][http_server_boss][T#1]{New I/O server boss #9}"
daemon [_thread_in_native, id=14787,
stack(0x00007f8051e4c000,0x00007f8051e8d000)]
0x00007f804c2c9000 JavaThread
"elasticsearch[Argus][http_server_worker][T#2]{New I/O worker #8}" daemon
[_thread_in_native, id=14786, stack(0x00007f8051e8d000,0x00007f8051ece000)]
0x00007f804c2c7800 JavaThread
"elasticsearch[Argus][http_server_worker][T#1]{New I/O worker #7}" daemon
[_thread_in_native, id=14785, stack(0x00007f8051ece000,0x00007f8051f0f000)]
0x0000000001b93000 JavaThread
"elasticsearch[Argus][riverClusterService#updateTask][T#1]" daemon
[_thread_blocked, id=14784, stack(0x00007f8051f0f000,0x00007f8051f50000)]
0x0000000001b94800 JavaThread
"elasticsearch[Argus][transport_client_timer][T#1]{Hashed wheel timer #1}"
daemon [_thread_blocked, id=14783,
stack(0x00007f8051f50000,0x00007f8051f91000)]
0x0000000001ef5000 JavaThread
"elasticsearch[Argus][clusterService#updateTask][T#1]" daemon
[_thread_blocked, id=14782, stack(0x00007f8051f91000,0x00007f8051fd2000)]
0x00007f804c78c800 JavaThread "elasticsearch[Argus][generic][T#1]" daemon
[_thread_blocked, id=14780, stack(0x00007f8052013000,0x00007f8052054000)]
0x00007f804c78b000 JavaThread
"elasticsearch[Argus][discovery#multicast#receiver][T#1]" daemon
[_thread_in_native, id=14779, stack(0x00007f8052054000,0x00007f8052095000)]
0x00007f804c75c000 JavaThread
"elasticsearch[Argus][transport_server_boss][T#1]{New I/O server boss #6}"
daemon [_thread_in_native, id=14778,
stack(0x00007f8052099000,0x00007f80520da000)]
0x00007f804c72d000 JavaThread
"elasticsearch[Argus][transport_server_worker][T#2]{New I/O worker #5}"
daemon [_thread_in_native, id=14777,
stack(0x00007f80520da000,0x00007f805211b000)]
0x00007f804c730000 JavaThread
"elasticsearch[Argus][transport_server_worker][T#1]{New I/O worker #4}"
daemon [_thread_in_native, id=14776,
stack(0x00007f805211b000,0x00007f805215c000)]
0x00007f804c724000 JavaThread
"elasticsearch[Argus][transport_client_boss][T#1]{New I/O boss #3}" daemon
[_thread_in_native, id=14775, stack(0x00007f805215c000,0x00007f805219d000)]
0x00007f804c6ed000 JavaThread
"elasticsearch[Argus][transport_client_worker][T#2]{New I/O worker #2}"
daemon [_thread_in_native, id=14774,
stack(0x00007f805219d000,0x00007f80521de000)]
0x00007f804c6d8800 JavaThread
"elasticsearch[Argus][transport_client_worker][T#1]{New I/O worker #1}"
daemon [_thread_in_native, id=14773,
stack(0x00007f80521de000,0x00007f805221f000)]
0x00007f804c6e7000 JavaThread "elasticsearch[Argus][[ttl_expire]]" daemon
[_thread_blocked, id=14772, stack(0x00007f805221f000,0x00007f8052260000)]
0x00007f804c68e800 JavaThread "elasticsearch[Argus][scheduler][T#1]"
daemon [_thread_in_Java, id=14771,
stack(0x00007f8052260000,0x00007f80522a1000)]
0x00007f804c4ab000 JavaThread "elasticsearch[Argus][[timer]]" daemon
[_thread_blocked, id=14770, stack(0x00007f80522a1000,0x00007f80522e2000)]
0x0000000001996000 JavaThread "Service Thread" daemon [_thread_blocked,
id=14768, stack(0x00007f8052c8e000,0x00007f8052ccf000)]
=>0x000000000198b000 JavaThread "C2 CompilerThread1" daemon [_thread_in_vm,
id=14767, stack(0x00007f8052ccf000,0x00007f8052dd0000)]
0x0000000001988800 JavaThread "C2 CompilerThread0" daemon [_thread_in_vm,
id=14766, stack(0x00007f8052dd0000,0x00007f8052ed1000)]
0x0000000001986000 JavaThread "Signal Dispatcher" daemon
[_thread_blocked, id=14765, stack(0x00007f8052ed1000,0x00007f8052f12000)]
0x0000000001984000 JavaThread "Surrogate Locker Thread (Concurrent GC)"
daemon [_thread_blocked, id=14764,
stack(0x00007f805ca47000,0x00007f805ca88000)]
0x0000000001936800 JavaThread "Finalizer" daemon [_thread_blocked,
id=14763, stack(0x00007f8052f12000,0x00007f8052f53000)]
0x0000000001934000 JavaThread "Reference Handler" daemon
[_thread_blocked, id=14762, stack(0x00007f8052f53000,0x00007f8052f94000)]
Other Threads:
0x000000000192c800 VMThread [stack:
0x00007f8052f94000,0x00007f8053095000] [id=14761]
0x00000000019a1000 WatcherThread [stack:
0x00007f8052b8d000,0x00007f8052c8e000] [id=14769]
VM state:not at safepoint (normal execution)
VM Mutex/Monitor currently owned by a thread: ([mutex/lock_event])
[0x0000000001874d90] CodeCache_lock - owner thread: 0x000000000198b000
[0x0000000001876510] Compile_lock - owner thread: 0x000000000198b000
[0x0000000001876610] MethodCompileQueue_lock - owner thread:
0x000000000198b000
Heap
def new generation total 76672K, used 66539K [0x00000000bae00000,
0x00000000c0130000, 0x00000000c0130000)
eden space 68160K, 85% used [0x00000000bae00000, 0x00000000be6aaed8,
0x00000000bf090000)
from space 8512K, 100% used [0x00000000bf090000, 0x00000000bf8e0000,
0x00000000bf8e0000)
to space 8512K, 0% used [0x00000000bf8e0000, 0x00000000bf8e0000,
0x00000000c0130000)
concurrent mark-sweep generation total 176960K, used 10885K
[0x00000000c0130000, 0x00000000cae00000, 0x00000000fae00000)
concurrent-mark-sweep perm gen total 31040K, used 30889K
[0x00000000fae00000, 0x00000000fcc50000, 0x0000000100000000)
Card table byte_map: [0x00007f8056dbf000,0x00007f8056fe9000] byte_map_base:
0x00007f80567e8000
Polling page: 0x00007f805cc05000
Code Cache [0x00007f8056fe9000, 0x00007f8057259000, 0x00007f8059fe9000)
total_blobs=728 nmethods=328 adapters=350 free_code_cache=48084Kb
largest_free_block=49175552
Compilation events (10 events):
Event: 48.911 Thread 0x0000000001988800 338
org.apache.lucene.util.PriorityQueue::downHeap (138 bytes)
Event: 48.915 Thread 0x000000000198b000 339
org.apache.lucene.store.BufferedIndexInput::readBytes (257 bytes)
Event: 48.919 Thread 0x000000000198b000 nmethod 339 0x00007f8057094250 code
[0x00007f80570943e0, 0x00007f8057094898]
Event: 48.919 Thread 0x000000000198b000 340
org.apache.lucene.store.BufferedIndexInput::readBytes (9 bytes)
Event: 48.920 Thread 0x000000000198b000 nmethod 340 0x00007f80570a5190 code
[0x00007f80570a52e0, 0x00007f80570a5368]
Event: 48.920 Thread 0x000000000198b000 341
org.apache.lucene.util.BytesRef:: (41 bytes)
Event: 48.920 Thread 0x000000000198b000 nmethod 341 0x00007f80570a4f90 code
[0x00007f80570a50c0, 0x00007f80570a5138]
Event: 48.924 Thread 0x0000000001988800 nmethod 338 0x00007f80570a7d90 code
[0x00007f80570a7f20, 0x00007f80570a82f8]
Event: 97.689 Thread 0x0000000001988800 342
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask::compareTo
(9 bytes)
Event: 97.690 Thread 0x0000000001988800 nmethod 343 0x00007f8057093f50 code
[0x00007f80570940a0, 0x00007f8057094178]
GC Heap History (4 events):
Event: 2.561 GC heap before
{Heap before GC invocations=0 (full 0):
def new generation total 76672K, used 68160K [0x00000000bae00000,
0x00000000c0130000, 0x00000000c0130000)
eden space 68160K, 100% used [0x00000000bae00000, 0x00000000bf090000,
0x00000000bf090000)
from space 8512K, 0% used [0x00000000bf090000, 0x00000000bf090000,
0x00000000bf8e0000)
to space 8512K, 0% used [0x00000000bf8e0000, 0x00000000bf8e0000,
0x00000000c0130000)
concurrent mark-sweep generation total 176960K, used 0K
[0x00000000c0130000, 0x00000000cae00000, 0x00000000fae00000)
concurrent-mark-sweep perm gen total 21248K, used 17799K
[0x00000000fae00000, 0x00000000fc2c0000, 0x0000000100000000)
Event: 2.594 GC heap after
Heap after GC invocations=1 (full 0):
def new generation total 76672K, used 8512K [0x00000000bae00000,
0x00000000c0130000, 0x00000000c0130000)
eden space 68160K, 0% used [0x00000000bae00000, 0x00000000bae00000,
0x00000000bf090000)
from space 8512K, 100% used [0x00000000bf8e0000, 0x00000000c0130000,
0x00000000c0130000)
to space 8512K, 0% used [0x00000000bf090000, 0x00000000bf090000,
0x00000000bf8e0000)
concurrent mark-sweep generation total 176960K, used 6830K
[0x00000000c0130000, 0x00000000cae00000, 0x00000000fae00000)
concurrent-mark-sweep perm gen total 21248K, used 17799K
[0x00000000fae00000, 0x00000000fc2c0000, 0x0000000100000000)
}
Event: 6.980 GC heap before
{Heap before GC invocations=1 (full 0):
def new generation total 76672K, used 76672K [0x00000000bae00000,
0x00000000c0130000, 0x00000000c0130000)
eden space 68160K, 100% used [0x00000000bae00000, 0x00000000bf090000,
0x00000000bf090000)
from space 8512K, 100% used [0x00000000bf8e0000, 0x00000000c0130000,
0x00000000c0130000)
to space 8512K, 0% used [0x00000000bf090000, 0x00000000bf090000,
0x00000000bf8e0000)
concurrent mark-sweep generation total 176960K, used 6830K
[0x00000000c0130000, 0x00000000cae00000, 0x00000000fae00000)
concurrent-mark-sweep perm gen total 25280K, used 25115K
[0x00000000fae00000, 0x00000000fc6b0000, 0x0000000100000000)
Event: 7.024 GC heap after
Heap after GC invocations=2 (full 0):
def new generation total 76672K, used 8512K [0x00000000bae00000,
0x00000000c0130000, 0x00000000c0130000)
eden space 68160K, 0% used [0x00000000bae00000, 0x00000000bae00000,
0x00000000bf090000)
from space 8512K, 100% used [0x00000000bf090000, 0x00000000bf8e0000,
0x00000000bf8e0000)
to space 8512K, 0% used [0x00000000bf8e0000, 0x00000000bf8e0000,
0x00000000c0130000)
concurrent mark-sweep generation total 176960K, used 10885K
[0x00000000c0130000, 0x00000000cae00000, 0x00000000fae00000)
concurrent-mark-sweep perm gen total 25280K, used 25115K
[0x00000000fae00000, 0x00000000fc6b0000, 0x0000000100000000)
}
Deoptimization events (10 events):
Event: 7.381 Thread 0x00007f804c78c800 Uncommon trap -58 fr.pc
0x00007f80570e75b4
Event: 7.381 Thread 0x00007f804c78c800 Uncommon trap -58 fr.pc
0x00007f80570d5ac8
Event: 7.381 Thread 0x00007f804c78c800 Uncommon trap -34 fr.pc
0x00007f80570e7014
Event: 7.381 Thread 0x00007f804c78c800 Uncommon trap -58 fr.pc
0x00007f80570d5ac8
Event: 7.381 Thread 0x00007f804c78c800 Uncommon trap -58 fr.pc
0x00007f80570e75b4
Event: 7.381 Thread 0x00007f804c78c800 Uncommon trap -34 fr.pc
0x00007f80570e7014
Event: 7.381 Thread 0x00007f804c78c800 Uncommon trap -58 fr.pc
0x00007f80570e75b4
Event: 7.381 Thread 0x00007f804c78c800 Uncommon trap -34 fr.pc
0x00007f80570e7014
Event: 15.297 Thread 0x0000000001b87800 Uncommon trap -83 fr.pc
0x00007f80570bfe08
Event: 15.746 Thread 0x0000000001b86800 Uncommon trap -28 fr.pc
0x00007f805708c880
Internal exceptions (10 events):
Event: 48.906 Thread 0x0000000001f3c800 Threw 0x00000000be4473e0 at
/HUDSON/workspace/jdk7u21-2-build-linux-amd64-product/jdk7u21/hotspot/src/share/vm/prims/jvm.cpp:1166
Event: 48.906 Thread 0x0000000001f3c800 Threw 0x00000000be44e4b0 at
/HUDSON/workspace/jdk7u21-2-build-linux-amd64-product/jdk7u21/hotspot/src/share/vm/prims/jvm.cpp:1166
Event: 48.906 Thread 0x0000000001f3c800 Threw 0x00000000be4522e8 at
/HUDSON/workspace/jdk7u21-2-build-linux-amd64-product/jdk7u21/hotspot/src/share/vm/prims/jvm.cpp:1166
Event: 48.907 Thread 0x0000000001f3c800 Threw 0x00000000be4f6798 at
/HUDSON/workspace/jdk7u21-2-build-linux-amd64-product/jdk7u21/hotspot/src/share/vm/prims/jvm.cpp:1166
Event: 48.909 Thread 0x0000000001f3c800 Threw 0x00000000be4ff788 at
/HUDSON/workspace/jdk7u21-2-build-linux-amd64-product/jdk7u21/hotspot/src/share/vm/prims/jvm.cpp:1166
Event: 48.909 Thread 0x0000000001f3c800 Threw 0x00000000be503b08 at
/HUDSON/workspace/jdk7u21-2-build-linux-amd64-product/jdk7u21/hotspot/src/share/vm/prims/jvm.cpp:1166
Event: 48.920 Thread 0x0000000001f3c800 Threw 0x00000000be50a048 at
/HUDSON/workspace/jdk7u21-2-build-linux-amd64-product/jdk7u21/hotspot/src/share/vm/prims/jvm.cpp:1166
Event: 48.921 Thread 0x0000000001f3c800 Threw 0x00000000be50f510 at
/HUDSON/workspace/jdk7u21-2-build-linux-amd64-product/jdk7u21/hotspot/src/share/vm/prims/jvm.cpp:1166
Event: 48.922 Thread 0x0000000001f3c800 Threw 0x00000000be517df8 at
/HUDSON/workspace/jdk7u21-2-build-linux-amd64-product/jdk7u21/hotspot/src/share/vm/prims/jvm.cpp:1166
Event: 65.114 Thread 0x00007f804c78b000 Threw 0x00000000be5fc1e8 at
/HUDSON/workspace/jdk7u21-2-build-linux-amd64-product/jdk7u21/hotspot/src/share/vm/prims/jni.cpp:742
Events (10 events):
Event: 48.922 loading class 0x0000000001c4e0d0 done
Event: 48.930 Executing VM operation: BulkRevokeBias
Event: 48.931 Executing VM operation: BulkRevokeBias done
Event: 48.931 Executing VM operation: RevokeBias
Event: 48.931 Executing VM operation: RevokeBias done
Event: 49.027 Executing VM operation: RevokeBias
Event: 49.027 Executing VM operation: RevokeBias done
Event: 49.028 Executing VM operation: RevokeBias
Event: 49.028 Executing VM operation: RevokeBias done
Event: 97.691 Thread 0x0000000001988800 flushing nmethod 0x00007f80570d03d0

Dynamic libraries:
00400000-00401000 r-xp 00000000 fd:02 1794092
/usr/lib/jvm/java-7-oracle/bin/java
00600000-00601000 rw-p 00000000 fd:02 1794092
/usr/lib/jvm/java-7-oracle/bin/java
0186e000-02288000 rw-p 00000000 00:00 0
[heap]
bae00000-cae00000 rw-p 00000000 00:00 0
cae00000-fae00000 rw-p 00000000 00:00 0
fae00000-fcc50000 rw-p 00000000 00:00 0
fcc50000-100000000 rw-p 00000000 00:00 0
7f804c000000-7f804c7b1000 rw-p 00000000 00:00 0
7f804c7b1000-7f8050000000 ---p 00000000 00:00 0
7f8051bcc000-7f8051d07000 rw-p 00000000 00:00 0
7f8051d07000-7f8051d0a000 ---p 00000000 00:00 0
7f8051d0a000-7f8051d48000 rw-p 00000000 00:00 0
7f8051d48000-7f8051d4b000 ---p 00000000 00:00 0
7f8051d4b000-7f8051d89000 rw-p 00000000 00:00 0
7f8051d89000-7f8051d8c000 ---p 00000000 00:00 0
7f8051d8c000-7f8051dca000 rw-p 00000000 00:00 0
7f8051dca000-7f8051dcd000 ---p 00000000 00:00 0
7f8051dcd000-7f8051e0b000 rw-p 00000000 00:00 0
7f8051e0b000-7f8051e0e000 ---p 00000000 00:00 0
7f8051e0e000-7f8051e4c000 rw-p 00000000 00:00 0
7f8051e4c000-7f8051e4f000 ---p 00000000 00:00 0
7f8051e4f000-7f8051e8d000 rw-p 00000000 00:00 0
7f8051e8d000-7f8051e90000 ---p 00000000 00:00 0
7f8051e90000-7f8051ece000 rw-p 00000000 00:00 0
7f8051ece000-7f8051ed1000 ---p 00000000 00:00 0
7f8051ed1000-7f8051f0f000 rw-p 00000000 00:00 0
7f8051f0f000-7f8051f12000 ---p 00000000 00:00 0
7f8051f12000-7f8051f50000 rw-p 00000000 00:00 0
7f8051f50000-7f8051f53000 ---p 00000000 00:00 0
7f8051f53000-7f8051f91000 rw-p 00000000 00:00 0
7f8051f91000-7f8051f94000 ---p 00000000 00:00 0
7f8051f94000-7f8051fd2000 rw-p 00000000 00:00 0
7f8051fd2000-7f8051fd5000 ---p 00000000 00:00 0
7f8051fd5000-7f8052013000 rw-p 00000000 00:00 0
7f8052013000-7f8052016000 ---p 00000000 00:00 0
7f8052016000-7f8052054000 rw-p 00000000 00:00 0
7f8052054000-7f8052057000 ---p 00000000 00:00 0
7f8052057000-7f8052095000 rw-p 00000000 00:00 0
7f8052095000-7f8052099000 r--s 0007d000 fd:02 1795816
/usr/lib/jvm/java-7-oracle/lib/jsse.jar
7f8052099000-7f805209c000 ---p 00000000 00:00 0
7f805209c000-7f80520da000 rw-p 00000000 00:00 0
7f80520da000-7f80520dd000 ---p 00000000 00:00 0
7f80520dd000-7f805211b000 rw-p 00000000 00:00 0
7f805211b000-7f805211e000 ---p 00000000 00:00 0
7f805211e000-7f805215c000 rw-p 00000000 00:00 0
7f805215c000-7f805215f000 ---p 00000000 00:00 0
7f805215f000-7f805219d000 rw-p 00000000 00:00 0
7f805219d000-7f80521a0000 ---p 00000000 00:00 0
7f80521a0000-7f80521de000 rw-p 00000000 00:00 0
7f80521de000-7f80521e1000 ---p 00000000 00:00 0
7f80521e1000-7f805221f000 rw-p 00000000 00:00 0
7f805221f000-7f8052222000 ---p 00000000 00:00 0
7f8052222000-7f8052260000 rw-p 00000000 00:00 0
7f8052260000-7f8052263000 ---p 00000000 00:00 0
7f8052263000-7f80522a1000 rw-p 00000000 00:00 0
7f80522a1000-7f80522a4000 ---p 00000000 00:00 0
7f80522a4000-7f80522e2000 rw-p 00000000 00:00 0
7f80522e2000-7f8052311000 r-xp 00000000 fd:02 1532116
/usr/share/elasticsearch/lib/sigar/libsigar-amd64-linux.so
7f8052311000-7f8052410000 ---p 0002f000 fd:02 1532116
/usr/share/elasticsearch/lib/sigar/libsigar-amd64-linux.so
7f8052410000-7f8052416000 rw-p 0002e000 fd:02 1532116
/usr/share/elasticsearch/lib/sigar/libsigar-amd64-linux.so
7f8052416000-7f8052419000 rw-p 00000000 00:00 0
7f8052419000-7f8052429000 r-xp 00000000 fd:02 1795744
/usr/lib/jvm/java-7-oracle/lib/amd64/libnio.so
7f8052429000-7f8052629000 ---p 00010000 fd:02 1795744
/usr/lib/jvm/java-7-oracle/lib/amd64/libnio.so
7f8052629000-7f805262a000 rw-p 00010000 fd:02 1795744
/usr/lib/jvm/java-7-oracle/lib/amd64/libnio.so
7f805262a000-7f805263f000 r-xp 00000000 fd:02 1795749
/usr/lib/jvm/java-7-oracle/lib/amd64/libnet.so
7f805263f000-7f805283f000 ---p 00015000 fd:02 1795749
/usr/lib/jvm/java-7-oracle/lib/amd64/libnet.so
7f805283f000-7f8052840000 rw-p 00015000 fd:02 1795749
/usr/lib/jvm/java-7-oracle/lib/amd64/libnet.so
7f8052840000-7f8052848000 r-xp 00000000 fd:02 1795796
/usr/lib/jvm/java-7-oracle/lib/amd64/libmanagement.so
7f8052848000-7f8052a47000 ---p 00008000 fd:02 1795796
/usr/lib/jvm/java-7-oracle/lib/amd64/libmanagement.so
7f8052a47000-7f8052a48000 rw-p 00007000 fd:02 1795796
/usr/lib/jvm/java-7-oracle/lib/amd64/libmanagement.so
7f8052a48000-7f8052a4f000 r--s 00062000 fd:02 1532132
/usr/share/elasticsearch/lib/sigar/sigar-1.6.4.jar
7f8052a4f000-7f8052a74000 r--s 001f8000 fd:02 1532140
/usr/share/elasticsearch/lib/lucene-core-4.3.1.jar
7f8052a74000-7f8052a78000 r--s 0001b000 fd:02 1532143
/usr/share/elasticsearch/lib/lucene-highlighter-4.3.1.jar
7f8052a78000-7f8052a7e000 r--s 0002c000 fd:02 1532149
/usr/share/elasticsearch/lib/lucene-queries-4.3.1.jar
7f8052a7e000-7f8052a87000 r--s 00056000 fd:02 1532193
/usr/share/elasticsearch/lib/lucene-queryparser-4.3.1.jar
7f8052a87000-7f8052a8a000 r--s 00014000 fd:02 1532195
/usr/share/elasticsearch/lib/lucene-spatial-4.3.1.jar
7f8052a8a000-7f8052a9a000 r--s 0016e000 fd:02 1532136
/usr/share/elasticsearch/lib/lucene-analyzers-common-4.3.1.jar
7f8052a9a000-7f8052a9e000 r--s 000d0000 fd:02 1532133
/usr/share/elasticsearch/lib/jna-3.3.0.jar
7f8052a9e000-7f8052aaf000 r--s 000ac000 fd:02 1532134
/usr/share/elasticsearch/lib/jts-1.12.jar
7f8052aaf000-7f8052ab5000 r--s 00043000 fd:02 1532138
/usr/share/elasticsearch/lib/lucene-codecs-4.3.1.jar
7f8052ab5000-7f8052ab7000 r--s 0000e000 fd:02 1532145
/usr/share/elasticsearch/lib/lucene-join-4.3.1.jar
7f8052ab7000-7f8052b8d000 r--s 009a8000 fd:02 1532115
/usr/share/elasticsearch/lib/elasticsearch-0.90.2.jar
7f8052b8d000-7f8052b8e000 ---p 00000000 00:00 0
7f8052b8e000-7f8052c8e000 rw-p 00000000 00:00 0
7f8052c8e000-7f8052c91000 ---p 00000000 00:00 0
7f8052c91000-7f8052ccf000 rw-p 00000000 00:00 0
7f8052ccf000-7f8052cd2000 ---p 00000000 00:00 0
7f8052cd2000-7f8052dd0000 rw-p 00000000 00:00 0
7f8052dd0000-7f8052dd3000 ---p 00000000 00:00 0
7f8052dd3000-7f8052ed1000 rw-p 00000000 00:00 0
7f8052ed1000-7f8052ed4000 ---p 00000000 00:00 0
7f8052ed4000-7f8052f12000 rw-p 00000000 00:00 0
7f8052f12000-7f8052f15000 ---p 00000000 00:00 0
7f8052f15000-7f8052f53000 rw-p 00000000 00:00 0
7f8052f53000-7f8052f56000 ---p 00000000 00:00 0
7f8052f56000-7f8052f94000 rw-p 00000000 00:00 0
7f8052f94000-7f8052f95000 ---p 00000000 00:00 0
7f8052f95000-7f8053105000 rw-p 00000000 00:00 0
7f8053105000-7f80532c3000 r--s 031bf000 fd:02 1794098
/usr/lib/jvm/java-7-oracle/lib/rt.jar
7f80532c3000-7f80532eb000 rw-p 00000000 00:00 0
7f80532eb000-7f80532ec000 ---p 00000000 00:00 0
7f80532ec000-7f8056c3f000 rw-p 00000000 00:00 0
7f8056c3f000-7f8056dbf000 rw-p 00000000 00:00 0
7f8056dbf000-7f8056e3f000 rw-p 00000000 00:00 0
7f8056e3f000-7f8056fbf000 rw-p 00000000 00:00 0
7f8056fbf000-7f8056fcf000 rw-p 00000000 00:00 0
7f8056fcf000-7f8056fe8000 rw-p 00000000 00:00 0
7f8056fe8000-7f8056fe9000 rw-p 00000000 00:00 0
7f8056fe9000-7f8057259000 rwxp 00000000 00:00 0
7f8057259000-7f8059fe9000 rw-p 00000000 00:00 0
7f8059fe9000-7f805a003000 r-xp 00000000 fd:02 1795798
/usr/lib/jvm/java-7-oracle/lib/amd64/libzip.so
7f805a003000-7f805a203000 ---p 0001a000 fd:02 1795798
/usr/lib/jvm/java-7-oracle/lib/amd64/libzip.so
7f805a203000-7f805a204000 rw-p 0001a000 fd:02 1795798
/usr/lib/jvm/java-7-oracle/lib/amd64/libzip.so
7f805a204000-7f805a210000 r-xp 00000000 fd:02 6905598
/lib/libnss_files-2.11.1.so
7f805a210000-7f805a40f000 ---p 0000c000 fd:02 6905598
/lib/libnss_files-2.11.1.so
7f805a40f000-7f805a410000 r--p 0000b000 fd:02 6905598
/lib/libnss_files-2.11.1.so
7f805a410000-7f805a411000 rw-p 0000c000 fd:02 6905598
/lib/libnss_files-2.11.1.so
7f805a411000-7f805a41b000 r-xp 00000000 fd:02 6905601
/lib/libnss_nis-2.11.1.so
7f805a41b000-7f805a61a000 ---p 0000a000 fd:02 6905601
/lib/libnss_nis-2.11.1.so
7f805a61a000-7f805a61b000 r--p 00009000 fd:02 6905601
/lib/libnss_nis-2.11.1.so
7f805a61b000-7f805a61c000 rw-p 0000a000 fd:02 6905601
/lib/libnss_nis-2.11.1.so
7f805a61c000-7f805a633000 r-xp 00000000 fd:02 6905594
/lib/libnsl-2.11.1.so
7f805a633000-7f805a832000 ---p 00017000 fd:02 6905594
/lib/libnsl-2.11.1.so
7f805a832000-7f805a833000 r--p 00016000 fd:02 6905594
/lib/libnsl-2.11.1.so
7f805a833000-7f805a834000 rw-p 00017000 fd:02 6905594
/lib/libnsl-2.11.1.so
7f805a834000-7f805a836000 rw-p 00000000 00:00 0
7f805a836000-7f805a83e000 r-xp 00000000 fd:02 6905586
/lib/libnss_compat-2.11.1.so
7f805a83e000-7f805aa3d000 ---p 00008000 fd:02 6905586
/lib/libnss_compat-2.11.1.so
7f805aa3d000-7f805aa3e000 r--p 00007000 fd:02 6905586
/lib/libnss_compat-2.11.1.so
7f805aa3e000-7f805aa3f000 rw-p 00008000 fd:02 6905586
/lib/libnss_compat-2.11.1.so
7f805aa3f000-7f805aa68000 r-xp 00000000 fd:02 1795756
/usr/lib/jvm/java-7-oracle/lib/amd64/libjava.so
7f805aa68000-7f805ac68000 ---p 00029000 fd:02 1795756
/usr/lib/jvm/java-7-oracle/lib/amd64/libjava.so
7f805ac68000-7f805ac6a000 rw-p 00029000 fd:02 1795756
/usr/lib/jvm/java-7-oracle/lib/amd64/libjava.so
7f805ac6a000-7f805ac77000 r-xp 00000000 fd:02 1795743
/usr/lib/jvm/java-7-oracle/lib/amd64/libverify.so
7f805ac77000-7f805ae76000 ---p 0000d000 fd:02 1795743
/usr/lib/jvm/java-7-oracle/lib/amd64/libverify.so
7f805ae76000-7f805ae78000 rw-p 0000c000 fd:02 1795743
/usr/lib/jvm/java-7-oracle/lib/amd64/libverify.so
7f805ae78000-7f805ae7f000 r-xp 00000000 fd:02 6905587
/lib/librt-2.11.1.so
7f805ae7f000-7f805b07e000 ---p 00007000 fd:02 6905587
/lib/librt-2.11.1.so
7f805b07e000-7f805b07f000 r--p 00006000 fd:02 6905587
/lib/librt-2.11.1.so
7f805b07f000-7f805b080000 rw-p 00007000 fd:02 6905587
/lib/librt-2.11.1.so
7f805b080000-7f805b102000 r-xp 00000000 fd:02 6905582
/lib/libm-2.11.1.so
7f805b102000-7f805b301000 ---p 00082000 fd:02 6905582
/lib/libm-2.11.1.so
7f805b301000-7f805b302000 r--p 00081000 fd:02 6905582
/lib/libm-2.11.1.so
7f805b302000-7f805b303000 rw-p 00082000 fd:02 6905582
/lib/libm-2.11.1.so
7f805b303000-7f805bd4d000 r-xp 00000000 fd:02 1795759
/usr/lib/jvm/java-7-oracle/lib/amd64/server/libjvm.so
7f805bd4d000-7f805bf4c000 ---p 00a4a000 fd:02 1795759
/usr/lib/jvm/java-7-oracle/lib/amd64/server/libjvm.so
7f805bf4c000-7f805bff0000 rw-p 00a49000 fd:02 1795759
/usr/lib/jvm/java-7-oracle/lib/amd64/server/libjvm.so
7f805bff0000-7f805c02b000 rw-p 00000000 00:00 0
7f805c02b000-7f805c1a8000 r-xp 00000000 fd:02 6905599
/lib/libc-2.11.1.so
7f805c1a8000-7f805c3a7000 ---p 0017d000 fd:02 6905599
/lib/libc-2.11.1.so
7f805c3a7000-7f805c3ab000 r--p 0017c000 fd:02 6905599
/lib/libc-2.11.1.so
7f805c3ab000-7f805c3ac000 rw-p 00180000 fd:02 6905599
/lib/libc-2.11.1.so
7f805c3ac000-7f805c3b1000 rw-p 00000000 00:00 0
7f805c3b1000-7f805c3b3000 r-xp 00000000 fd:02 6905597
/lib/libdl-2.11.1.so
7f805c3b3000-7f805c5b3000 ---p 00002000 fd:02 6905597
/lib/libdl-2.11.1.so
7f805c5b3000-7f805c5b4000 r--p 00002000 fd:02 6905597
/lib/libdl-2.11.1.so
7f805c5b4000-7f805c5b5000 rw-p 00003000 fd:02 6905597
/lib/libdl-2.11.1.so
7f805c5b5000-7f805c5cb000 r-xp 00000000 fd:02 1795754
/usr/lib/jvm/java-7-oracle/lib/amd64/jli/libjli.so
7f805c5cb000-7f805c7cb000 ---p 00016000 fd:02 1795754
/usr/lib/jvm/java-7-oracle/lib/amd64/jli/libjli.so
7f805c7cb000-7f805c7cc000 rw-p 00016000 fd:02 1795754
/usr/lib/jvm/java-7-oracle/lib/amd64/jli/libjli.so
7f805c7cc000-7f805c7e4000 r-xp 00000000 fd:02 6905593
/lib/libpthread-2.11.1.so
7f805c7e4000-7f805c9e3000 ---p 00018000 fd:02 6905593
/lib/libpthread-2.11.1.so
7f805c9e3000-7f805c9e4000 r--p 00017000 fd:02 6905593
/lib/libpthread-2.11.1.so
7f805c9e4000-7f805c9e5000 rw-p 00018000 fd:02 6905593
/lib/libpthread-2.11.1.so
7f805c9e5000-7f805c9e9000 rw-p 00000000 00:00 0
7f805c9e9000-7f805ca09000 r-xp 00000000 fd:02 6905590
/lib/ld-2.11.1.so
7f805ca09000-7f805ca0c000 r--s 00020000 fd:02 1532196
/usr/share/elasticsearch/lib/lucene-suggest-4.3.1.jar
7f805ca0c000-7f805ca0f000 r--s 00018000 fd:02 1532142
/usr/share/elasticsearch/lib/lucene-grouping-4.3.1.jar
7f805ca0f000-7f805ca18000 r--s 0006f000 fd:02 1532135
/usr/share/elasticsearch/lib/log4j-1.2.17.jar
7f805ca18000-7f805ca47000 rw-p 00000000 00:00 0
7f805ca47000-7f805ca4a000 ---p 00000000 00:00 0
7f805ca4a000-7f805cad8000 rw-p 00000000 00:00 0
7f805cad8000-7f805caf2000 rw-p 00000000 00:00 0
7f805caf2000-7f805cafc000 rw-p 00000000 00:00 0
7f805cafc000-7f805cbb2000 rw-p 00000000 00:00 0
7f805cbb2000-7f805cbba000 rw-s 00000000 fd:02 4112385
/tmp/hsperfdata_root/14753
7f805cbba000-7f805cbbd000 ---p 00000000 00:00 0
7f805cbbd000-7f805cbff000 rw-p 00000000 00:00 0
7f805cbff000-7f805cc01000 r--s 0000a000 fd:02 1532194
/usr/share/elasticsearch/lib/lucene-sandbox-4.3.1.jar
7f805cc01000-7f805cc02000 r--s 00008000 fd:02 1532147
/usr/share/elasticsearch/lib/lucene-memory-4.3.1.jar
7f805cc02000-7f805cc04000 r--s 0000f000 fd:02 1532197
/usr/share/elasticsearch/lib/spatial4j-0.3.jar
7f805cc04000-7f805cc05000 rw-p 00000000 00:00 0
7f805cc05000-7f805cc06000 r--p 00000000 00:00 0
7f805cc06000-7f805cc08000 rw-p 00000000 00:00 0
7f805cc08000-7f805cc09000 r--p 0001f000 fd:02 6905590
/lib/ld-2.11.1.so
7f805cc09000-7f805cc0a000 rw-p 00020000 fd:02 6905590
/lib/ld-2.11.1.so
7f805cc0a000-7f805cc0b000 rw-p 00000000 00:00 0
7fff650d4000-7fff650e9000 rw-p 00000000 00:00 0
[stack]
7fff65105000-7fff65107000 r-xp 00000000 00:00 0
[vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0
[vsyscall]
VM Arguments:
jvm_args: -Xms256m -Xmx1g -Xss256k -Djava.awt.headless=true
-XX:+UseParNewGC -XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly
-XX:+HeapDumpOnOutOfMemoryError -Delasticsearch
-Des.pidfile=/var/run/elasticsearch.pid -Des.foreground=yes
-Des.path.home=/usr/share/elasticsearch
-Des.default.config=/etc/elasticsearch/elasticsearch.yml
-Des.default.path.home=/usr/share/elasticsearch
-Des.default.path.logs=/var/log/elasticsearch
-Des.default.path.data=/var/lib/elasticsearch
-Des.default.path.work=/tmp/elasticsearch
-Des.default.path.conf=/etc/elasticsearch
java_command: org.elasticsearch.bootstrap.Elasticsearch
Launcher Type: SUN_STANDARD
Environment Variables:
JAVA_HOME=/usr/lib/jvm/java-7-oracle
PATH=/bin:/usr/bin:/sbin:/usr/sbin
SHELL=/bin/bash
Signal Handlers:
SIGSEGV: [libjvm.so+0x8a80e0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGBUS: [libjvm.so+0x8a80e0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGFPE: [libjvm.so+0x744170], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGPIPE: [libjvm.so+0x744170], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGXFSZ: [libjvm.so+0x744170], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGILL: [libjvm.so+0x744170], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGUSR1: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGUSR2: [libjvm.so+0x743ac0], sa_mask[0]=0x00000000, sa_flags=0x10000004
SIGHUP: [libjvm.so+0x745e50], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGINT: [libjvm.so+0x745e50], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGTERM: [libjvm.so+0x745e50], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGQUIT: [libjvm.so+0x745e50], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004

--------------- S Y S T E M ---------------
OS:squeeze/sid
uname:Linux 2.6.32-6-pve #1 SMP Mon Jan 23 08:27:52 CET 2012 x86_64
libc:glibc 2.11.1 NPTL 2.11.1
rlimit: STACK 8192k, CORE 0k, NPROC infinity, NOFILE 1024, AS infinity
load average:0.00 0.00 0.00
/proc/meminfo:
MemTotal: 5242880 kB
MemFree: 3053144 kB
Cached: 539444 kB
Active: 960892 kB
Inactive: 165404 kB
Active(anon): 586844 kB
Inactive(anon): 8 kB
Active(file): 374048 kB
Inactive(file): 165396 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 52 kB
AnonPages: 586852 kB
Shmem: 5120 kB
Slab: 46352 kB
SReclaimable: 34004 kB
SUnreclaim: 12348 kB

CPU:total 1 (4 cores per cpu, 2 threads per core) family 6 model 44
stepping 2, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2,
popcnt, ht, tsc, tscinvbit, tscinv
/proc/cpuinfo:
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 44
model name : Intel(R) Xeon(R) CPU E5620 @ 2.40GHz
stepping : 2
cpu MHz : 2394.012
cache size : 12288 KB
physical id : 1
siblings : 8
core id : 0
cpu cores : 4
apicid : 32
initial apicid : 32
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov
pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology
nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2
ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt aes lahf_lm ida arat
tpr_shadow vnmi flexpriority ept vpid
bogomips : 4788.02
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:

Memory: 4k page, physical 5242880k(3053144k free), swap 0k(0k free)
vm_info: Java HotSpot(TM) 64-Bit Server VM (23.21-b01) for linux-amd64 JRE
(1.7.0_21-b11), built on Apr 4 2013 04:03:29 by "java_re" with gcc 4.3.0
20080428 (Red Hat 4.3.0-8)
time: Fri Jul 5 11:38:05 2013
elapsed time: 97 seconds

Thanks,
DJP

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

jprante · July 5, 2013, 3:48pm

There are many issues with Java on OpenVZ (if you ask google), so I
would try to run ES without OpenVZ to see if the issue persists.

The crash "ShouldNotReachHere()" means that Java can't find the correct
executable assembler code (within OpenVZ address space) so without
deeper knowledge of Java Assembler code for HotSpot VM and OpenVZ it is
not possible to fix this. If you prefer you should report this to OpenVZ
and/or Oracle Java, maybe they have more knowlegde about this kind of
crashes.

In the dump, there is "Memory: 4k page, physical 5242880k(3053144k
free), swap 0k(0k free)" which means you run without swap. This may be
harmless or not.

Jörg

Am 05.07.13 11:50, schrieb DJP:

Hello Jörg,

Many thanks for your answer. You're right about the dump, my mistake...

Here's one with Java 7u21 and ES 0.90.2. Furthermore, i've checked my
swap (1024M) and raised up to 2048M. (I'm in an openvz environment...)

#
[thread 140189111027456 also had an error]# A fatal error has been
detected by the Java Runtime Environment:
#
#  Internal Error (assembler_x86.cpp:746), pid=14753,
tid=140189122754304
#  Error: ShouldNotReachHere()
#
# JRE version: 7.0_21-b11
# Java VM: Java HotSpot(TM) 64-Bit Server VM (23.21-b01 mixed mode
linux-amd64 compressed oops)
# Failed to write core dump. Core dumps have been disabled. To
enable core dumping, try "ulimit -c unlimited" before starting
Java again
#
# An error report file with more information is saved as:
# /usr/lib/jvm/hs_err_pid14753.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.sun.com/bugreport/crash.jsp
#
Aborted

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.