TIBCO RV high CPU hangs. BE-ENGINE

Symptoms:

Be-engine consuming around 25 percent of the CPU and messages are not getting through TIBCO. One thread of BE-ENGINE running ~4% CPU and another one ~20%

Some of the pointers to the “possibly related” issues:

http://weblogs.asp.net/mdavey/archive/2004/03/10/87220.aspx

https://wiki.mst.edu/deskinst/change/server/licenseservers/20090112_licshared01_qla_ooc

Quite verbose thread stack listing is at the bottom. But most importantly, what we found pointed us in the direction of upgrading TIBCO machines from jre 1.5 to 1.6u10+.

Currently, TIBCO admin guys are trying to let one of the TIBCO instances to run on 1.6u16 which turns out to be somehow a none trivial task. Will be back with results.

0:000> ~* e kb2000 ChildEBP RetAddr  Args to Child              WARNING: Stack unwind information not available. Following frames may be wrong. 0012fd08 77e61c8d 00002eec ffffffff 00000000 ntdll!KiFastSystemCallRet *** ERROR: Symbol file could not be found.  Defaulted to export symbols for jvm.dll - 0012fd1c 6d815d1b 00002eec ffffffff 00019bc0 kernel32!WaitForSingleObject+0x12 0012fd44 6d8536c3 00000000 ffffffff 00000000 jvm!jmm_GetLastGCStat+0x69b0 00000000 00000000 00000000 00000000 00000000 jvm!JVM_RegisterPerfMethods+0x2fcce ChildEBP RetAddr  Args to Child           X threads with the following stack:   WARNING: Stack unwind information not available. Following frames may be wrong. 16c3fc78 77e61c8d 00002e20 ffffffff 00000000 ntdll!KiFastSystemCallRet 16c3fc8c 6d815cda 00002e20 ffffffff 1a5d20f8 kernel32!WaitForSingleObject+0x12 16c3fcb4 6d7aea5e 00000001 ffffffff 00000000 jvm!jmm_GetLastGCStat+0x696f 16c3fcd0 6d7af0f6 00000000 00000000 00a86258 jvm!AsyncGetCallTrace+0x46b9 16c3fd4c 6d81d261 808b4900 905bda50 80a5c456 jvm!AsyncGetCallTrace+0x4d51 *** ERROR: Symbol file could not be found.  Defaulted to export symbols for msvcrt.dll - 16c3ff84 77bcb530 00a063d0 00000000 00000000 jvm!jmm_GetLastGCStat+0xdef6 16c3ffb8 77e64829 00a06d40 00000000 00000000 msvcrt!endthreadex+0xa3 16c3ffec 00000000 77bcb4bc 00a06d40 00000000 kernel32!GetModuleHandleA+0xdf ChildEBP RetAddr  Args to Child               ………  X, with a slight variation of the above: ChildEBP RetAddr  Args to Child              WARNING: Stack unwind information not available. Following frames may be wrong. 1703fcc4 6d794e46 110f8430 6d829d22 0001fc90 jvm!AsyncGetCallTrace+0x119da 1703fccc 6d829d22 0001fc90 6d8be950 1703fd2c jvm+0x54e46 1703fce0 6d82992d 0ec4fff0 00000001 0001fc90 jvm!JVM_RegisterPerfMethods+0x632d 1703fd2c 6d82908e 00000009 00a0df10 00000000 jvm!JVM_RegisterPerfMethods+0x5f38 1703fe18 6d82dfbb 1a17f3e0 00000000 1a17f3c4 jvm!JVM_RegisterPerfMethods+0x5699 1703fe38 6d822b5f 1a17f3e0 00a0df10 1a17f3c4 jvm!JVM_RegisterPerfMethods+0xa5c6 1703fe60 6d86af2b 1a17f3e0 0000000c 00000000 jvm!jmm_GetLastGCStat+0x137f4 1703fe8c 6d86ace1 1a17f3c4 00a85828 00a85840 jvm!JVM_RegisterUnsafeMethods+0xe693 1703fea8 6d86a7af 00019bf0 00000000 00000000 jvm!JVM_RegisterUnsafeMethods+0xe449 1703feec 6d86a958 1a17f3c4 0001a238 000144f8 jvm!JVM_RegisterUnsafeMethods+0xdf17 1703ff40 6d86a6bd 00000000 00a081e0 6d81d261 jvm!JVM_RegisterUnsafeMethods+0xe0c0 1703ff84 77bcb530 000144f8 00000000 00000000 jvm!JVM_RegisterUnsafeMethods+0xde25 1703ffb8 77e64829 00a861c0 00000000 00000000 msvcrt!endthreadex+0xa3 1703ffec 00000000 77bcb4bc 00a861c0 00000000 kernel32!GetModuleHandleA+0xdf ChildEBP RetAddr  Args to Child                   Then, something more interesting: *** ERROR: Symbol file could not be found.  Defaulted to export symbols for mswsock.dll - ChildEBP RetAddr  Args to Child              WARNING: Stack unwind information not available. Following frames may be wrong. 188ffde0 71b21a03 00002810 0000282c 00000000 ntdll!KiFastSystemCallRet *** ERROR: Symbol file could not be found.  Defaulted to export symbols for ws2_32.dll - 188ffed0 71c0283c 0000282d 186bb150 00000000 mswsock+0x1a03 *** WARNING: Unable to verify checksum for tibrv.dll *** ERROR: Symbol file could not be found.  Defaulted to export symbols for tibrv.dll - 188fff20 184bdad3 0000282d 186bb150 00000000 ws2_32!select+0xa1 188fff68 184bd92c 186bb370 188fff84 184d8046 tibrv!tibrvBuffer_Expand+0x5903 188fff74 184d8046 00000000 186bb668 188fffb8 tibrv!tibrvBuffer_Expand+0x575c *** ERROR: Symbol file could not be found.  Defaulted to export symbols for msvcr71.dll - 188fff84 7c349565 186bb668 00000000 00000000 tibrv!tibrv_Rand+0x416 188fffb8 77e64829 186bb678 00000000 00000000 msvcr71!endthreadex+0xa0 188fffec 00000000 7c3494f6 186bb678 00000000 kernel32!GetModuleHandleA+0xdf ChildEBP RetAddr  Args to Child              WARNING: Stack unwind information not available. Following frames may be wrong. 189ffe68 77e61c8d 00002840 00004adc 00000000 ntdll!KiFastSystemCallRet 189ffe7c 184d7f3a 00002840 00004adc 184ef4f0 kernel32!WaitForSingleObject+0x12 189ffe90 184d87c1 186bb638 184ef4f0 e8000000 tibrv!tibrv_Rand+0x30a 189ffeb0 184bd57e 186bb638 184ef4f0 e8000000 tibrv!tibrvCondition_TimedWait+0x61 189fff68 184bd48c 186bb468 189fff84 184d8046 tibrv!tibrvBuffer_Expand+0x53ae 189fff74 184d8046 00000000 186bb728 189fffb8 tibrv!tibrvBuffer_Expand+0x52bc 189fff84 7c349565 186bb728 00000000 00000000 tibrv!tibrv_Rand+0x416 189fffb8 77e64829 186bb738 00000000 00000000 msvcr71!endthreadex+0xa0 189fffec 00000000 7c3494f6 186bb738 00000000 kernel32!GetModuleHandleA+0xdf ChildEBP RetAddr  Args to Child              WARNING: Stack unwind information not available. Following frames may be wrong. 18affde0 71b21a03 00002808 000027ec 00000000 ntdll!KiFastSystemCallRet 18affed0 71c0283c 000027ed 186cb008 00000000 mswsock+0x1a03 18afff20 184bd7eb 000027ed 186cb008 00000000 ws2_32!select+0xa1 18afff74 184d8046 00000000 186bb7e8 18afffb8 tibrv!tibrvBuffer_Expand+0x561b 18afff84 7c349565 186bb7e8 00000000 00000000 tibrv!tibrv_Rand+0x416 18afffb8 77e64829 186bb7f8 00000000 00000000 msvcr71!endthreadex+0xa0 18afffec 00000000 7c3494f6 186bb7f8 00000000 kernel32!GetModuleHandleA+0xdf ChildEBP RetAddr  Args to Child              WARNING: Stack unwind information not available. Following frames may be wrong. 18bffe80 77e61c8d 00002818 ffffffff 00000000 ntdll!KiFastSystemCallRet 18bffe94 184d7ead 00002818 ffffffff 186bb8a4 kernel32!WaitForSingleObject+0x12 18bffea8 184d8758 186bb958 186bb8a4 00000000 tibrv!tibrv_Rand+0x27d 18bffec0 18491c43 186bb958 186bb8a4 00000000 tibrv!tibrvCondition_Wait+0x58 18bffed8 184c75df 186bb890 186bb958 00000000 tibrv!tibrvObjectId_Wait+0x43 18bfff38 184c7a3f 186bb890 00000000 bff00000 tibrv!tibrvMsg_CreateFromBytes+0x1ff 18bfff58 184c813e 00000004 00000000 bff00000 tibrv!tibrvQueue_TimedDispatch+0x7f 18bfff74 184d8046 000003ea 186bba28 18bfffb8 tibrv!tibrvDispatcher_CreateEx+0x23e 18bfff84 7c349565 186bba28 00000000 00000000 tibrv!tibrv_Rand+0x416 18bfffb8 77e64829 186bba38 00000000 00000000 msvcr71!endthreadex+0xa0 18bfffec 00000000 7c3494f6 186bba38 00000000 kernel32!GetModuleHandleA+0xdf ChildEBP RetAddr  Args to Child              WARNING: Stack unwind information not available. Following frames may be wrong. 1908f8b4 77e61c8d 00002538 ffffffff 00000000 ntdll!KiFastSystemCallRet 1908f8c8 184d7ead 00002538 ffffffff 18769384 kernel32!WaitForSingleObject+0x12 1908f8dc 184d8758 18704250 18769384 1908f8fc tibrv!tibrv_Rand+0x27d 1908f8f4 18491c43 18704250 18769384 00000000 tibrv!tibrvCondition_Wait+0x58 1908f90c 184c75df 18769370 18704250 00000000 tibrv!tibrvObjectId_Wait+0x43 1908f96c 184c7a3f 18769370 00000000 bff00000 tibrv!tibrvMsg_CreateFromBytes+0x1ff *** WARNING: Unable to verify checksum for tibrvjsd.dll *** ERROR: Symbol file could not be found.  Defaulted to export symbols for tibrvjsd.dll - 1908f98c 184514d7 0000040a 00000000 bff00000 tibrv!tibrvQueue_TimedDispatch+0x7f 1908fa38 6d7c73ed 1908fa6c 1908fc10 0000000a tibrvjsd!Java_com_tibco_tibrv_TibrvImplQueueC_natDispatch+0x27 ChildEBP RetAddr  Args to Child              WARNING: Stack unwind information not available. Following frames may be wrong. 1977fb34 77e61c8d 0000210c ffffffff 00000000 ntdll!KiFastSystemCallRet 1977fb48 184d7ead 0000210c ffffffff 186e4284 kernel32!WaitForSingleObject+0x12 1977fb5c 184d8758 186b43e0 186e4284 1977fb7c tibrv!tibrv_Rand+0x27d 1977fb74 18491c43 186b43e0 186e4284 00000000 tibrv!tibrvCondition_Wait+0x58 1977fb8c 184c75df 186e4270 186b43e0 00000000 tibrv!tibrvObjectId_Wait+0x43 1977fbec 184c7a3f 186e4270 00000000 bff00000 tibrv!tibrvMsg_CreateFromBytes+0x1ff 1977fc0c 184514d7 0000041e 00000000 bff00000 tibrv!tibrvQueue_TimedDispatch+0x7f 1977fcb8 6d7c73ed 1977fcec 1977fe90 0000000a tibrvjsd!Java_com_tibco_tibrv_TibrvImplQueueC_natDispatch+0x27 1977fd34 6d81fb96 0000000a 00000000 1977fde8 jvm!AsyncGetCallTrace+0x1d048 1977fd78 6d7c72be 6d7c72c2 1977fe88 1977fd9c jvm!jmm_GetLastGCStat+0x1082b 1977fdc8 6d7c701b 1977fe88 17835164 6d8c4594 jvm!AsyncGetCallTrace+0x1cf19 1977fe44 6d7e1e95 1977fe88 17835160 17835164 jvm!AsyncGetCallTrace+0x1cc76 1977fe98 6d851325 180c9f50 180c9f50 180c9f50 jvm!JVM_StartThread+0x186 1977fec4 6d8512f3 178a3460 6d81d261 00000000 jvm!JVM_RegisterPerfMethods+0x2d930 1977ff84 77bcb530 180c9f50 00000000 00000000 jvm!JVM_RegisterPerfMethods+0x2d8fe 1977ffb8 77e64829 17af8ce0 00000000 00000000 msvcrt!endthreadex+0xa3 1977ffec 00000000 77bcb4bc 17af8ce0 00000000 kernel32!GetModuleHandleA+0xdf ChildEBP RetAddr  Args to Child              WARNING: Stack unwind information not available. Following frames may be wrong. 1997f9d8 77e61c8d 000020cc ffffffff 00000000 ntdll!KiFastSystemCallRet 1997f9ec 6d81a40a 000020cc ffffffff 6d81a284 kernel32!WaitForSingleObject+0x12 1997fa50 6d842374 00000000 00000000 00000001 jvm!jmm_GetLastGCStat+0xb09f 1997fa6c 6d7dd625 17b30544 00000000 00000000 jvm!JVM_RegisterPerfMethods+0x1e97f 1997faa0 00aa82ff 17b309e8 1997faec 00000000 jvm!JVM_MonitorWait+0x80 1997fbb8 6d7c73ed 1997fbec 1997fd90 0000000a 0xaa82ff 1997fc34 6d81fb96 0000000a 00000000 1997fce8 jvm!AsyncGetCallTrace+0x1d048 1997fc78 6d7c72be 6d7c72c2 1997fd88 1997fc9c jvm!jmm_GetLastGCStat+0x1082b 1997fcc8 6d7c701b 1997fd88 17b30534 6d8c4594 jvm!AsyncGetCallTrace+0x1cf19 1997fd44 6d7e1e95 1997fd88 17b30530 17b30534 jvm!AsyncGetCallTrace+0x1cc76 1997fd98 6d851325 17b30928 17b30928 17b30928 jvm!JVM_StartThread+0x186 1997fdc4 6d8512f3 17aea0f8 6d81d261 00000000 jvm!JVM_RegisterPerfMethods+0x2d930 1997ff84 77bcb530 17b30928 00000000 00000000 jvm!JVM_RegisterPerfMethods+0x2d8fe 1997ffb8 77e64829 17af8ce0 00000000 00000000 msvcrt!endthreadex+0xa3 1997ffec 00000000 77bcb4bc 17af8ce0 00000000 kernel32!GetModuleHandleA+0xdf ChildEBP RetAddr  Args to Child              WARNING: Stack unwind information not available. Following frames may be wrong. 19a7f958 77e61c8d 00002080 ffffffff 00000000 ntdll!KiFastSystemCallRet 19a7f96c 6d81a40a 00002080 ffffffff 6d81a284 kernel32!WaitForSingleObject+0x12 19a7f9d0 6d842374 00000000 00000000 00000001 jvm!jmm_GetLastGCStat+0xb09f 19a7f9ec 6d7dd625 181d0acc 00000000 00000000 jvm!JVM_RegisterPerfMethods+0x1e97f 19a7fa20 00aa82ff 181ea6d8 19a7fa6c 00000000 jvm!JVM_MonitorWait+0x80 19a7fb38 6d7c73ed 19a7fb6c 19a7fd10 0000000a 0xaa82ff 19a7fbb4 6d81fb96 0000000a 00000000 19a7fc68 jvm!AsyncGetCallTrace+0x1d048 19a7fbf8 6d7c72be 6d7c72c2 19a7fd08 19a7fc1c jvm!jmm_GetLastGCStat+0x1082b 19a7fc48 6d7c701b 19a7fd08 181d0abc 6d8c4594 jvm!AsyncGetCallTrace+0x1cf19 19a7fcc4 6d7e1e95 19a7fd08 181d0ab8 181d0abc jvm!AsyncGetCallTrace+0x1cc76 19a7fd18 6d851325 181ea618 181ea618 181ea618 jvm!JVM_StartThread+0x186 19a7fd44 6d8512f3 181d02e0 6d81d261 00000000 jvm!JVM_RegisterPerfMethods+0x2d930 19a7ff84 77bcb530 181ea618 00000000 00000000 jvm!JVM_RegisterPerfMethods+0x2d8fe 19a7ffb8 77e64829 17af8ce0 00000000 00000000 msvcrt!endthreadex+0xa3 19a7ffec 00000000 77bcb4bc 17af8ce0 00000000 kernel32!GetModuleHandleA+0xdf ChildEBP RetAddr  Args to Child              WARNING: Stack unwind information not available. Following frames may be wrong. 19b7f8d0 77e61c8d 00002d90 ffffffff 00000000 ntdll!KiFastSystemCallRet 19b7f8e4 6d81a5e1 00002d90 ffffffff 00a88f34 kernel32!WaitForSingleObject+0x12 19b7f904 6d81a2e0 00000001 00000000 17b4a9dc jvm!jmm_GetLastGCStat+0xb276 19b7f94c 6d842374 00000000 00000000 00000001 jvm!jmm_GetLastGCStat+0xaf75 19b7f968 6d7dd625 17b4a9dc 00000000 00000000 jvm!JVM_RegisterPerfMethods+0x1e97f 19b7f99c 00aa82ff 17afd7f8 19b7f9e8 00000000 jvm!JVM_MonitorWait+0x80 19b7fab8 6d7c73ed 19b7faec 19b7fc90 0000000a 0xaa82ff 19b7fb34 6d81fb96 0000000a 00000000 19b7fbe8 jvm!AsyncGetCallTrace+0x1d048 19b7fb78 6d7c72be 6d7c72c2 19b7fc88 19b7fb9c jvm!jmm_GetLastGCStat+0x1082b 19b7fbc8 6d7c701b 19b7fc88 17b4a9cc 6d8c4594 jvm!AsyncGetCallTrace+0x1cf19 19b7fc44 6d7e1e95 19b7fc88 17b4a9c8 17b4a9cc jvm!AsyncGetCallTrace+0x1cc76 19b7fc98 6d851325 17afd738 17afd738 17afd738 jvm!JVM_StartThread+0x186 19b7fcc4 6d8512f3 181e5d58 6d81d261 00000000 jvm!JVM_RegisterPerfMethods+0x2d930 19b7ff84 77bcb530 17afd738 00000000 00000000 jvm!JVM_RegisterPerfMethods+0x2d8fe 19b7ffb8 77e64829 17af8ce0 00000000 00000000 msvcrt!endthreadex+0xa3 19b7ffec 00000000 77bcb4bc 17af8ce0 00000000 kernel32!GetModuleHandleA+0xdf ChildEBP RetAddr  Args to Child              WARNING: Stack unwind information not available. Following frames may be wrong. 19c7f858 77e61c8d 00002034 ffffffff 00000000 ntdll!KiFastSystemCallRet 19c7f86c 6d81a40a 00002034 ffffffff 6d81a284 kernel32!WaitForSingleObject+0x12 19c7f8d0 6d842374 00000000 00000000 00000001 jvm!jmm_GetLastGCStat+0xb09f 19c7f8ec 6d7dd625 17b4ca0c 00000000 00000000 jvm!JVM_RegisterPerfMethods+0x1e97f 19c7f920 00aa82ff 17b4d4c0 19c7f96c 00000000 jvm!JVM_MonitorWait+0x80 19c7fa38 6d7c73ed 19c7fa6c 19c7fc10 0000000a 0xaa82ff 19c7fab4 6d81fb96 0000000a 00000000 19c7fb68 jvm!AsyncGetCallTrace+0x1d048 19c7faf8 6d7c72be 6d7c72c2 19c7fc08 19c7fb1c jvm!jmm_GetLastGCStat+0x1082b 19c7fb48 6d7c701b 19c7fc08 17b4c9fc 6d8c4594 jvm!AsyncGetCallTrace+0x1cf19 19c7fbc4 6d7e1e95 19c7fc08 17b4c9f8 17b4c9fc jvm!AsyncGetCallTrace+0x1cc76 19c7fc18 6d851325 17b4d400 17b4d400 17b4d400 jvm!JVM_StartThread+0x186 19c7fc44 6d8512f3 181e5df0 6d81d261 00000000 jvm!JVM_RegisterPerfMethods+0x2d930 19c7ff84 77bcb530 17b4d400 00000000 00000000 jvm!JVM_RegisterPerfMethods+0x2d8fe 19c7ffb8 77e64829 17af8ce0 00000000 00000000 msvcrt!endthreadex+0xa3 19c7ffec 00000000 77bcb4bc 17af8ce0 00000000 kernel32!GetModuleHandleA+0xdf ChildEBP RetAddr  Args to Child              WARNING: Stack unwind information not available. Following frames may be wrong. 19e7f184 71b2c517 00002028 00001fb0 00000000 ntdll!KiFastSystemCallRet 19e7f1fc 71c02fee 00001fb0 19e7f234 00000001 mswsock!ServiceMain+0x1e49 *** ERROR: Symbol file could not be found.  Defaulted to export symbols for net.dll - 19e7f244 6d546a10 00001fb0 1a752000 00008000 ws2_32!recv+0x6f 19e7fb58 6d74b189 19e7fb94 00000000 019db1de net!Java_java_net_SocketInputStream_socketRead0+0x140 19e7fb6c 6d81d472 670efab9 002c8ff1 00002710 jvm+0xb189 19e7fb98 6d781e44 00000000 19e7fbcc 00aa2a8f jvm!jmm_GetLastGCStat+0xe107 19e7fb9c 00000000 19e7fbcc 00aa2a8f 124b7790 jvm+0x41e44 ChildEBP RetAddr  Args to Child              WARNING: Stack unwind information not available. Following frames may be wrong. 19f7f084 71b2c517 00001fa4 00001f84 00000000 ntdll!KiFastSystemCallRet 19f7f0fc 71c02fee 00001f84 19f7f134 00000001 mswsock!ServiceMain+0x1e49 19f7f144 6d546a10 00001f84 182b7008 00008000 ws2_32!recv+0x6f 19f7fa58 6d74b189 19f7fa94 00000000 019db1de net!Java_java_net_SocketInputStream_socketRead0+0x140 19f7fa6c 6d81d472 6a2688bd 002c8ff1 00002710 jvm+0xb189 19f7fa98 6d781e44 00000000 19f7facc 00aa2a8f jvm!jmm_GetLastGCStat+0xe107 19f7fa9c 00000000 19f7facc 00aa2a8f 124c0cd8 jvm+0x41e44 ChildEBP RetAddr  Args to Child              WARNING: Stack unwind information not available. Following frames may be wrong. 1a07ef84 71b2c517 00001f78 00001f58 00000000 ntdll!KiFastSystemCallRet 1a07effc 71c02fee 00001f58 1a07f034 00000001 mswsock!ServiceMain+0x1e49 1a07f044 6d546a10 00001f58 18264008 00008000 ws2_32!recv+0x6f 1a07f958 6d74b189 1a07f994 00000000 019db1de net!Java_java_net_SocketInputStream_socketRead0+0x140 1a07f96c 6d81d472 670efab9 002c8ff1 00002710 jvm+0xb189 1a07f998 6d781e44 00000000 1a07f9cc 00aa2a8f jvm!jmm_GetLastGCStat+0xe107 1a07f99c 00000000 1a07f9cc 00aa2a8f 12484ca8 jvm+0x41e44 ChildEBP RetAddr  Args to Child              WARNING: Stack unwind information not available. Following frames may be wrong. 1a17f334 77e61c8d 00002edc ffffffff 00000000 ntdll!KiFastSystemCallRet 1a17f348 6d815d1b 00002edc ffffffff 1a17f3c4 kernel32!WaitForSingleObject+0x12 1a17f370 6d86ab52 00000000 ffffffff 00000000 jvm!jmm_GetLastGCStat+0x69b0 1a17f394 6d86aecb 00001f23 00000000 00019c20 jvm!JVM_RegisterUnsafeMethods+0xe2ba 00000000 00000000 00000000 00000000 00000000 jvm!JVM_RegisterUnsafeMethods+0xe633 ChildEBP RetAddr  Args to Child              WARNING: Stack unwind information not available. Following frames may be wrong. 1a27f204 71b2c517 00001f38 00001f18 00000000 ntdll!KiFastSystemCallRet 1a27f27c 71c02fee 00001f18 1a27f2b4 00000001 mswsock!ServiceMain+0x1e49 1a27f2c4 6d546a10 00001f18 1829f008 00008000 ws2_32!recv+0x6f 1a27fbd8 6d74b189 1a27fc14 00000000 019db1de net!Java_java_net_SocketInputStream_socketRead0+0x140 1a27fbec 6d81d472 670efab9 002c8ff1 00002710 jvm+0xb189 1a27fc18 6d781e44 00000000 1a27fc4c 00aa2a8f jvm!jmm_GetLastGCStat+0xe107 1a27fc1c 00000000 1a27fc4c 00aa2a8f 12484800 jvm+0x41e44 ChildEBP RetAddr  Args to Child              WARNING: Stack unwind information not available. Following frames may be wrong. 1a37fa9c 77e61c8d 00001f04 ffffffff 00000000 ntdll!KiFastSystemCallRet 1a37fab0 6d81a40a 00001f04 ffffffff 6d81a284 kernel32!WaitForSingleObject+0x12 1a37fb14 6d842374 00000000 00000000 00000001 jvm!jmm_GetLastGCStat+0xb09f 1a37fb30 6d7dd625 1813ac5c 00000000 00000000 jvm!JVM_RegisterPerfMethods+0x1e97f 1a37fb64 00aa82ff 17aa8ef0 1a37fbb0 00000000 jvm!JVM_MonitorWait+0x80 1a37fb98 00aa29e3 00000000 00aa6509 00000000 0xaa82ff 1a37fcb8 6d7c73ed 1a37fcec 1a37fe90 0000000a 0xaa29e3 1a37fd34 6d81fb96 0000000a 00000000 1a37fde8 jvm!AsyncGetCallTrace+0x1d048 1a37fd78 6d7c72be 6d7c72c2 1a37fe88 1a37fd9c jvm!jmm_GetLastGCStat+0x1082b 1a37fdc8 6d7c701b 1a37fe88 1813ac4c 6d8c4594 jvm!AsyncGetCallTrace+0x1cf19 1a37fe44 6d7e1e95 1a37fe88 1813ac48 1813ac4c jvm!AsyncGetCallTrace+0x1cc76 1a37fe98 6d851325 17aa8e30 17aa8e30 17aa8e30 jvm!JVM_StartThread+0x186 1a37fec4 6d8512f3 18161bf0 6d81d261 00000000 jvm!JVM_RegisterPerfMethods+0x2d930 1a37ff84 77bcb530 17aa8e30 00000000 00000000 jvm!JVM_RegisterPerfMethods+0x2d8fe 1a37ffb8 77e64829 17b289b0 00000000 00000000 msvcrt!endthreadex+0xa3 1a37ffec 00000000 77bcb4bc 17b289b0 00000000 kernel32!GetModuleHandleA+0xdf ChildEBP RetAddr  Args to Child              WARNING: Stack unwind information not available. Following frames may be wrong. 1a47f084 71b2c517 00001ef8 00001ed8 00000000 ntdll!KiFastSystemCallRet 1a47f0fc 71c02fee 00001ed8 1a47f134 00000001 mswsock!ServiceMain+0x1e49 1a47f144 6d546a10 00001ed8 1a749ff8 00008000 ws2_32!recv+0x6f 1a47fa58 6d74b189 1a47fa94 00000000 019db1de net!Java_java_net_SocketInputStream_socketRead0+0x140 1a47fa6c 6d81d472 6a2688bd 002c8ff1 00002710 jvm+0xb189 1a47fa98 6d781e44 00000000 1a47facc 00aa2a8f jvm!jmm_GetLastGCStat+0xe107 1a47fa9c 00000000 1a47facc 00aa2a8f 1248a6c0 jvm+0x41e44