Exclude SetClientMode test in FIPS 140-3 excludes #1071
Conversation
|
Please add some detail explaining why this particular test should be excluded. |
|
This test fails as follows : There are numerous tests failing with this message since we intentionally disable setting of these properties programmatically when in various FIPS modes. |
jasonkatonica
force-pushed
the
katonica/issue/fipsexcludes
branch
2 times, most recently
from
43fcf1c to
0bad8f1
Compare
|
Could we not just exclude the two subtests that try to modify if ("TLSv1".equals(protocol) || "TLSv1.1".equals(protocol)) {
SecurityUtils.removeFromDisabledTlsAlgs(protocol);
} |
In the long run we do want to modify the TLS tests in a more comprehensive way similar to what you are suggesting. Today very very few of the TLS tests are running including this one. We are still talking about reviving this effort to finish PR #891 and enable as many tests as possible however it is a bigger size effort to get the tests working correctly. Chances even if we change the code just for this one test the crypto in the test will cause a failure when running in FIPS mode. |
|
I wasn't proposing any test code changes; I was just suggesting that rather than exclude all four subtests, we only exclude the two that fail; that is instead of If you know that the other two subtests fail, please describe the problem they encounter. |
|
When this runs in the test buckets such as Which has the relevant test failure like: I don't see any of the other subtests attempting to run once it hits this issue to see if they pass or fail. Maybe subtests can be run somehow that i am not aware of. |
|
Did you run that test in the context of my suggestion? If not, it's conceivable that the framework stops at the first subtest failure. |
|
I gave this a try today with Java 21 using this jenkins job. This run made use of this set of exclude changes. Note that this uses the and |
|
With this in the problem list: I would not expect to even attempt running The changes you reference (ibmruntimes/openj9-openjdk-jdk21@openj9...jasonkatonica:openj9-openjdk-jdk21:katonica/issue/fipsexcludes) are for jdk21 which will have no bearing on this repository. |
|
Correct I am only running this there since Java 21 is more stable then building JDKs and running tests / builds on next. The same framework for test execution should be in place there. |
|
Perhaps the exclusion |
Do you happen to have an example in a test that uses a |
|
Perhaps we can use |
I can try this. Id assume that |
|
This seems to be confirmed by the jtreg FAQ. |
|
I gave this a try using the syntax |
|
Ok, we tried. I guess for now we can just exclude the whole class. |
|
Please rebase to avoid implicit merges (e.g. with #1073). |
This test fails as follows : ``` Execution failed: `main' threw exception: java.lang.SecurityException: Property 'jdk.tls.disabledAlgorithms' cannot be set programmatically when in FIPS mode ``` This test should be excluded accordingly. Signed-off-by: Jason Katonica <katonica@us.ibm.com>
jasonkatonica
force-pushed
the
katonica/issue/fipsexcludes
branch
from
0bad8f1 to
08e3d31
Compare
|
Rebased this PR |
2 participants
This test fails as follows :
This test should be excluded accordingly.
Signed-off-by: Jason Katonica katonica@us.ibm.com