APM Java agent websphere startup failure


(D1le) #1

Hello!
I have some troubles running websphere server with apm agent.

Env:
RedHat 7.5
WebSphere ND 8.5.5.12 cluster configuration with one node
Java 1.8 IBM J9
APM server 6.4.2
APM java agent 0.9

When I use agent, websphere server fails to run throwing an exception java.lang.ClassNotFoundException: org.omg.CORBA.ORB
Are there any ideas, how to fix this?


(D1le) #2

org.eclipse.core.runtime.CoreException: Plug-in "com.ibm.ws.runtime" was unable to instantiate class "com.ibm.ws.runtime.WsServer".
at org.eclipse.core.internal.registry.osgi.RegistryStrategyOSGI.throwException(RegistryStrategyOSGI.java:194)
at org.eclipse.core.internal.registry.osgi.RegistryStrategyOSGI.createExecutableExtension(RegistryStrategyOSGI.java:188)
at org.eclipse.core.internal.registry.ExtensionRegistry.createExecutableExtension(ExtensionRegistry.java:904)
at org.eclipse.core.internal.registry.ConfigurationElement.createExecutableExtension(ConfigurationElement.java:243)
at org.eclipse.core.internal.registry.ConfigurationElementHandle.createExecutableExtension(ConfigurationElementHandle.java:55)
at com.ibm.wsspi.bootstrap.WSLauncher.getClassFromExtension(WSLauncher.java:246)
at com.ibm.wsspi.bootstrap.WSLauncher.launchMain(WSLauncher.java:185)
at com.ibm.wsspi.bootstrap.WSLauncher.main(WSLauncher.java:96)
at com.ibm.wsspi.bootstrap.WSLauncher.run(WSLauncher.java:77)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:508)
at org.eclipse.equinox.internal.app.EclipseAppContainer.callMethodWithException(EclipseAppContainer.java:587)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:198)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:369)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:508)
at org.eclipse.core.launcher.Main.invokeFramework(Main.java:340)
at org.eclipse.core.launcher.Main.basicRun(Main.java:282)
at org.eclipse.core.launcher.Main.run(Main.java:981)
at com.ibm.wsspi.bootstrap.WSPreLauncher.launchEclipse(WSPreLauncher.java:406)
at com.ibm.wsspi.bootstrap.WSPreLauncher.main(WSPreLauncher.java:169)
Caused by: java.lang.ExceptionInInitializerError
at java.lang.J9VMInternals.ensureError(J9VMInternals.java:141)
at java.lang.J9VMInternals.recordInitializationFailure(J9VMInternals.java:130)
at java.lang.J9VMInternals.newInstanceImpl(Native Method)
at java.lang.Class.newInstance(Class.java:1899)
at org.eclipse.core.internal.registry.osgi.RegistryStrategyOSGI.createExecutableExtension(RegistryStrategyOSGI.java:184)
... 26 more
Caused by: java.lang.IllegalStateException: java.lang.NoClassDefFoundError: org.omg.CORBA.ORB
at com.ibm.ws.util.ImplFactory.loadImplFromClass(ImplFactory.java:355)
at com.ibm.ws.util.ImplFactory.loadImplFromKey(ImplFactory.java:329)
at com.ibm.ws.util.ImplFactory.loadImplFromKey(ImplFactory.java:333)
at com.ibm.ws.runtime.WsServer.(WsServer.java:51)
... 29 more
Caused by: java.lang.NoClassDefFoundError: org.omg.CORBA.ORB
at sun.misc.Unsafe.ensureClassInitialized(Native Method)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:82)
at java.lang.Class.forName(Class.java:287)
at com.ibm.ws.util.PlatformHelperFactory.getBackupHelper(PlatformHelperFactory.java:127)
at com.ibm.ws.util.PlatformHelperFactory.getPlatformHelper(PlatformHelperFactory.java:93)
at com.ibm.ws.runtime.WsServerImpl.(WsServerImpl.java:156)
at java.lang.J9VMInternals.newInstanceImpl(Native Method)
at java.lang.Class.newInstance(Class.java:1899)
at com.ibm.ws.util.ImplFactory.loadImplFromClass(ImplFactory.java:350)
... 32 more
Caused by: java.lang.ClassNotFoundException: org.omg.CORBA.ORB
at org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:460)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:422)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:410)
at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loadClass(DefaultClassLoader.java:107)
at java.lang.ClassLoader.loadClass(ClassLoader.java:809)
... 41 more


(D1le) #3

2018-10-24 12:39:29.388 [main] INFO co.elastic.apm.configuration.StartupInfo - Starting Elastic APM 0.9.0-SNAPSHOT as prpc on Java 1.8.0 (IBM Corporation) Linux 3.10.0-862.3.2.el7.x86_64
2018-10-24 12:39:29.392 [main] DEBUG co.elastic.apm.configuration.StartupInfo - service_name: 'prpc' (source: /u01/IBM/AppServer/lib/ext/elasticapm.properties)
2018-10-24 12:39:29.393 [main] DEBUG co.elastic.apm.configuration.StartupInfo - server_urls: 'http://10.116.175.165:8200' (source: /u01/IBM/AppServer/lib/ext/elasticapm.properties)
2018-10-24 12:39:29.393 [main] DEBUG co.elastic.apm.configuration.StartupInfo - log_level: 'TRACE' (source: /u01/IBM/AppServer/lib/ext/elasticapm.properties)
2018-10-24 12:39:29.393 [main] WARN co.elastic.apm.configuration.StartupInfo - To enable all features and to increase startup times, please configure application_packages
2018-10-24 12:39:29.424 [apm-server-healthcheck] INFO co.elastic.apm.report.ApmServerHealthChecker - Elastic APM server is available
2018-10-24 12:39:29.445 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.plugin.api.ElasticApmApiInstrumentation$StartTransactionInstrumentation
2018-10-24 12:39:29.453 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.plugin.api.ElasticApmApiInstrumentation$CurrentTransactionInstrumentation
2018-10-24 12:39:29.454 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.plugin.api.ElasticApmApiInstrumentation$CurrentSpanInstrumentation
2018-10-24 12:39:29.454 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.plugin.api.ElasticApmApiInstrumentation$CaptureExceptionInstrumentation
2018-10-24 12:39:29.455 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.plugin.api.TransactionInstrumentation$SetNameInstrumentation
2018-10-24 12:39:29.455 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.plugin.api.TransactionInstrumentation$SetTypeInstrumentation
2018-10-24 12:39:29.455 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.plugin.api.TransactionInstrumentation$AddTagInstrumentation
2018-10-24 12:39:29.455 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.plugin.api.TransactionInstrumentation$SetUserInstrumentation
2018-10-24 12:39:29.456 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.plugin.api.TransactionInstrumentation$EndInstrumentation
2018-10-24 12:39:29.456 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.plugin.api.TransactionInstrumentation$DoCreateSpanInstrumentation
2018-10-24 12:39:29.456 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.plugin.api.TransactionInstrumentation$CaptureExceptionInstrumentation
2018-10-24 12:39:29.456 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.plugin.api.TransactionInstrumentation$GetIdInstrumentation
2018-10-24 12:39:29.456 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.plugin.api.SpanInstrumentation$SetNameInstrumentation
2018-10-24 12:39:29.456 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.plugin.api.SpanInstrumentation$SetTypeInstrumentation
2018-10-24 12:39:29.457 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.plugin.api.SpanInstrumentation$DoCreateSpanInstrumentation
2018-10-24 12:39:29.457 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.plugin.api.SpanInstrumentation$EndInstrumentation
2018-10-24 12:39:29.457 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.plugin.api.SpanInstrumentation$CaptureExceptionInstrumentation
2018-10-24 12:39:29.457 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.plugin.api.SpanInstrumentation$GetIdInstrumentation
2018-10-24 12:39:29.457 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.plugin.api.SpanInstrumentation$AddTagInstrumentation
2018-10-24 12:39:29.458 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.plugin.api.CaptureExceptionInstrumentation
2018-10-24 12:39:29.458 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.jaxrs.JaxRsTransactionNameInstrumentation
2018-10-24 12:39:29.458 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.jdbc.ConnectionInstrumentation
2018-10-24 12:39:29.459 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.jdbc.StatementInstrumentation
2018-10-24 12:39:29.459 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.jdbc.PreparedStatementInstrumentation
2018-10-24 12:39:29.459 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.opentracing.impl.ApmSpanInstrumentation$FinishInstrumentation
2018-10-24 12:39:29.460 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.opentracing.impl.ApmSpanInstrumentation$CreateErrorInstrumentation
2018-10-24 12:39:29.460 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.opentracing.impl.ApmSpanInstrumentation$TagInstrumentation
2018-10-24 12:39:29.460 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.opentracing.impl.ApmSpanInstrumentation$SetOperationName
2018-10-24 12:39:29.460 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.opentracing.impl.ApmSpanInstrumentation$BaggageItemsInstrumentation
2018-10-24 12:39:29.460 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.opentracing.impl.ApmSpanBuilderInstrumentation$CreateSpanInstrumentation
2018-10-24 12:39:29.461 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.opentracing.impl.ScopeManagerInstrumentation$ActivateInstrumentation
2018-10-24 12:39:29.461 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.opentracing.impl.ScopeManagerInstrumentation$CurrentSpanInstrumentation
2018-10-24 12:39:29.461 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.opentracing.impl.ApmScopeInstrumentation
2018-10-24 12:39:29.461 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.servlet.ServletInstrumentation
2018-10-24 12:39:29.462 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.servlet.FilterChainInstrumentation
2018-10-24 12:39:29.462 [main] DEBUG co.elastic.apm.bci.ElasticApmAgent - Applying advice co.elastic.apm.servlet.AsyncInstrumentation


(Felix Barnsteiner) #4

Hi and thanks for giving the Java agent a try.

This seems like a very strange error and I can't quite explain the cause. The Java agent does not use and mess with CORBA classes. Does your application make use of it? Does that error only happen with a specific application or which any you have tried so far? Was the exception thrown before or after the agent has started up? Were there any more Caused bys in the exception stack trace? I can't see any agent class in the stacktrace (co.elastic.apm) which makes this quite hard to track down.

Note that we only test against WebSphere Liberty

Cheers,
Felix


(D1le) #5

Hello! Thank you for reply! I'll try to get more info about the issue and bring it here


(system) #6

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.