DebugSqlSpec and integration tests

Elastic Stack Elasticsearch
1

Hi,

I can't make the DebugSqlSpec test class work.

When I run an Elasticsearch local server (./gradlew run --debug-jvm -Drun.license_type=trial -Dxpack.security.enabled=false) and then run an empty implementation of the DebugSqlSpec test class against this local instance, I got the error below:

java.security.AccessControlException: access denied ("java.lang.reflect.ReflectPermission" "suppressAccessChecks")
	at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
	at java.base/java.security.AccessController.checkPermission(AccessController.java:895)
	at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
	at java.base/java.lang.reflect.AccessibleObject.checkPermission(AccessibleObject.java:83)
	at java.base/java.lang.reflect.Field.setAccessible(Field.java:169)
	at com.intellij.junit4.JUnit4IdeaTestRunner.getSuiteMethodDescription(JUnit4IdeaTestRunner.java:139)
	at com.intellij.junit4.JUnit4IdeaTestRunner.getDescription(JUnit4IdeaTestRunner.java:84)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:50)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

I tried the idea.no.launcher=true trick, but no luck.

Integration tests with only no-arg constructors seem to work fine, but the DebugSqlSpec has a constructor with arguments because it's a parametrized test.

Could an Elastic insider explain the typical SQL support development workflow (reproduce / debug / fix / test) please?

Thanks a lot,
Chris

2

I tend to add the test to the spec file that it feels best in and then run:

./gradlew -p x-pack/plugin/sql/qa/single-node/ integTest -Dtests.class='*SpecIT'

I don't tend to use the DebugSpecs. You can also use -Dtests.method='*some*glob*' to limit which test cases are run which is nice for keeping the output more organized. The process of starting gradle and starting the single node makes the whole thing kind of heavy but it is the burden that comes with integration testing.

3

Thank you @nik9000, are you then able to debug the started node instance?

Maybe with tweaked ES_JAVA_OPTS?

4

I don't typically debug the instance but this looks like it works:

ES_JAVA_OPTS="-Xdebug -Xrunjdwp:server=y,transport=dt_socket,address=4000,suspend=y" ./gradlew -p x-pack/plugin/sql/qsingle-node/ integTest -Dtests.class='*SpecIT'

Gradle pauses with

:x-pack:plugin:sql:qa:single-node:integTestCluster#wait

up. I think it'll wait 30 seconds or something before giving up. That really isn't very long to be honest.

5

That is great, thank you @nik9000, I’ll give it a try tomorrow and let you know.

6

Nope, unfortunately, it didn't work.

Attempt 1

I've also tried with a shell export, but still no luck.

export ES_JAVA_OPTS="-Xdebug -Xrunjdwp:server=y,transport=dt_socket,address=8000,suspend=y"

./gradlew -p x-pack/plugin/sql/qa/single-node/ integTest -Dtests.class='*DebugSqlSpecIT'

And the remote debugging output is still:

Error running 'remote': Unable to open debugger port (localhost:8000): java.net.ConnectException "Connection refused (Connection refused)"

Attempt 2

I've tried to tweak Gradle build file x-pack/plugin/sql/qa/single-node/build.gradle by adding:

integTestCluster {
  setting 'xpack.security.enabled', 'false'
  setting 'xpack.license.self_generated.type', 'trial'
  
  String jvmArgsStr = " -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=8000"
  jvmArgs jvmArgsStr
}

Then I can connect the remote debugger, but the Gradle integration test task waits for 30 seconds and run the test without halting on my breakpoint.

Attempt 3

I tried another JVM argument string variant, but still no luck.

integTestCluster {
  setting 'xpack.security.enabled', 'false'
  setting 'xpack.license.self_generated.type', 'trial'
  
  String jvmArgsStr = " -Xdebug -Xrunjdwp:server=y,transport=dt_socket,address=8000,suspend=y"
  jvmArgs jvmArgsStr
}

Do you usually use a debugger to troubleshoot x-pack issues or do you use some kind of print statements?

7

For the most part I user unit tests and log messages. It has been about a year since I connected a debugger to Elasticsearch. I'm unsure what happened here. When I did it on the command line it looked like it stopped and was waiting for me but I didn't try to connect because I didn't have Eclipse running at the time.

8

That’s okay, thank you @nik9000 :+1: I will launch a local instance with Gradle and --debug-jvm option, load the test_emp data set and use cURL, as usual, it will be fine.

9

I've just seen this main entry point org.elasticsearch.xpack.sql.qa.jdbc.DataLoader#main, looks awesome :wink:

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.