Test cases failing with seed option to gradle

[vitt-bagal]

Hello,

Actually I'm running the test cases for Elasticsearch 6.1.1. I have used below command to run a failed test case on ubuntu 17.10.

gradle :modules:reindex:test -Dtests.seed=694995B964450E90 -
Dtests.class=org.elasticsearch.index.reindex.CancelTests -Dtests.method="testReindexCancel" -Dtests.security.manager=true -Dtests.locale=id-ID -Dtests.timezone=Australia/Currie

which is returning below error and test is getting failed

FAILURE 62.1s | CancelTests.testReindexCancel <<< FAILURES!

Throwable #1: java.lang.AssertionError: expected:<0> but was:<32>
at __randomizedtesting.SeedInfo.seed([C3B139A89FBD9D2D:6D8E5340D3C48EC2]:0)
at org.elasticsearch.index.reindex.CancelTests.lambda$testCancel$2(CancelTests.java:157)
at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:726)
at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:700)
at org.elasticsearch.index.reindex.CancelTests.testCancel(CancelTests.java:153)
at org.elasticsearch.index.reindex.CancelTests.testReindexCancel(CancelTests.java:200)
at java.lang.Thread.run(Thread.java:748)
Suppressed: java.lang.AssertionError: expected:<0> but was:<32>
at org.elasticsearch.index.reindex.CancelTests.lambda$testCancel$2(CancelTests.java:157)
at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:714)
... 40 more

Also I observed that , the test is passing without seed option , see below command-
gradle :modules:reindex:test -Dtests.class=org.elasticsearch.index.reindex.CancelTests -Dtests.method="testReindexCancel" -Dtests.security.manager=true -Dtests.locale=id-ID -Dtests.timezone=Australia/Currie

So I just want to know what is the exact role of seed option ? If the test is passing without seed option then Is it ok to ignore the test failure occurred with seed ?

Please help me out.

Thanks.

[dadoonet (David Pilato)]

Tests are randomized. The seed helps to reproduce the test issue using the same parameters it was running with the first time.

No seed provide = a new seed is generated = params are generated using that seed
Seed provided = params are generated using that seed

[rjernst (Ryan Ernst)]

If the test is passing without seed option then Is it ok to ignore the test failure occurred with seed ?

That depends. If you run that repro line on master without your change, and it still fails, then it should be safe to ignore (and an issue should be filed with the repro line).