Migrate OIDC/SAML broker config-variant and infrastructure-dependent tests to the new testsuite by sguilhen · Pull Request #51949 · keycloak/keycloak

sguilhen · 2026-08-21T18:49:20Z

Follow-up to #51859, which landed the shared broker test infrastructure(BrokerConfigSupport, OidcBrokerConfigSupport, SamlBrokerConfigSupport,BrokerLoginTest). This PR migrates the next batch of broker tests from the legacy Arquillian testsuite to the new JUnit 5 framework.

What's included

36 broker test classes migrated across OIDC and SAML:
- OIDC config-variant and parameter-forwarding tests (auth methods, PKCE, ui-locales, IdP hint / nonce, sub-match introspection, etc.)
- JWE and ECDH-ES token-encryption variants
- Infrastructure-dependent tests: custom-provider JWKS (missing-use, unsupported-key), private-key-JWT client auth, vault-backed IdP secret, max_age propagation (time-offset), and KcSamlSignedDocumentOnlyBrokerTest
4 custom SPI providers ported to tests/custom-providers (JWKS resource providers, client-id-required JWT authenticator, test OIDC IdP factory)
Package reorganization: broker tests split into oidc/ and saml/ subpackages; shared support interfaces kept at the package root
FIPS coverage (follow-up): the 10 JWE/ECDH-ES tests are not added to the FIPS suites in this PR. Under FIPS the server runs with TLS on, and the outbound broker callback currently fails the TLS handshake because the test framework doesn't wire the server's self-signed cert into the outbound truststore - a framework gap, not a crypto issue. Restoring FIPS coverage for these tests is tracked separately.

The legacy test classes are retained where they still carry test methods not yet covered by this migration.

Testing

All migrated broker tests pass locally (Base2TestSuite picks up the new subpackages automatically via recursive @SelectPackages).

Closes #51955

Copilot

Pull request overview

Migrates OIDC/SAML broker scenarios from Arquillian to the JUnit 5 test framework and ports their supporting providers.

Changes:

Migrates 36 broker configuration and infrastructure-dependent tests.
Adds JWE/ECDH configuration support and custom broker providers.
Reorganizes tests into OIDC/SAML packages and removes migrated legacy coverage.

Reviewed changes

Copilot reviewed 97 out of 97 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
`testsuite/integration-arquillian/tests/base/testsuites/fips-suite`	Removes deferred JWE FIPS cases.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcSamlSignedDocumentOnlyBrokerTest.java`	Removes legacy SAML test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerVaultTest.java`	Removes legacy vault test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerVaultConfiguration.java`	Removes legacy vault configuration.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerUiLocalesWithIdpHintTest.java`	Removes legacy locale/hint test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerUiLocalesEnabledTest.java`	Removes legacy locale-enabled test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerUiLocalesDisabledTest.java`	Removes legacy locale-disabled test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerSubMatchIntrospectionTest.java`	Removes legacy subject-mismatch test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerPromptParameterTest.java`	Removes legacy prompt test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerPrivateKeyJwtWithX509HeadersTest.java`	Removes legacy X.509 JWT test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerPrivateKeyJwtUnsupportedKeyTest.java`	Removes legacy unsupported-key test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerPrivateKeyJwtTest.java`	Removes legacy private-key JWT test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerPrivateKeyJwtMissingUseTest.java`	Removes legacy missing-use test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerPrivateKeyJwtCustomSignAlgTest.java`	Removes legacy custom-algorithm test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerPrivateKeyJwtCustomAudienceTest.java`	Removes legacy custom-audience test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerPrivateKeyJwtClientIdRequiredTest.java`	Removes legacy client-ID test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerPkceTest.java`	Removes legacy PKCE test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerPassMaxAgeTest.java`	Removes legacy max-age test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerParameterForwardTest.java`	Removes legacy forwarding test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerNonceParameterTest.java`	Removes legacy nonce test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerNoLoginHintTest.java`	Removes legacy login-hint test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerJWEUserInfoJustEncryptedTest.java`	Removes legacy encrypted-userinfo test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerJWEEcdhEsP384A192GcmTest.java`	Removes legacy JWE variant.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerJWEEcdhEsP384A192CbcHs384Test.java`	Removes legacy JWE variant.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerJWEEcdhEsA256KwP521A256GcmTest.java`	Removes legacy JWE variant.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerJWEEcdhEsA256KwP521A256CbcHs512Test.java`	Removes legacy JWE variant.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerJWEEcdhEsA192KwP384A192GcmTest.java`	Removes legacy JWE variant.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerJWEEcdhEsA192KwP384A192CbcHs384Test.java`	Removes legacy JWE variant.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerJWEEcdhEsA128KwP256A128GcmTest.java`	Removes legacy JWE variant.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerJWEEcdhEsA128KwP256A128CbcHs256Test.java`	Removes legacy JWE variant.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerIdpPublicKeyMissingUseTest.java`	Removes legacy JWKS test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerIdpHintTest.java`	Removes legacy IdP-hint test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerHiddenIdpHintTest.java`	Removes legacy hidden-IdP test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerColonAliasClientSecretBasicAuthTest.java`	Removes legacy colon-ID test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerClientSecretJwtTest.java`	Removes legacy secret-JWT test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerClientSecretJwtCustomSignAlgTest.java`	Removes legacy custom-HMAC test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/KcOidcBrokerAcrParameterTest.java`	Removes legacy ACR test.
`testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/broker/AbstractKcOidcBrokerJWEEcdhEsTest.java`	Removes legacy JWE support.
`tests/custom-providers/pom.xml`	Generates a Jandex provider index.
`tests/custom-providers/src/main/resources/META-INF/services/org.keycloak.services.resource.RealmResourceProviderFactory`	Registers JWKS providers.
`tests/custom-providers/src/main/resources/META-INF/services/org.keycloak.broker.provider.IdentityProviderFactory`	Registers test OIDC factory.
`tests/custom-providers/src/main/resources/META-INF/services/org.keycloak.authentication.ClientAuthenticatorFactory`	Registers custom authenticator.
`tests/custom-providers/src/main/java/org/keycloak/tests/providers/broker/oidc/UnsupportedKeyJwksRealmResourceProviderFactory.java`	Creates unsupported-key JWKS provider.
`tests/custom-providers/src/main/java/org/keycloak/tests/providers/broker/oidc/UnsupportedKeyJwksRealmResourceProvider.java`	Serves unsupported-key JWKS.
`tests/custom-providers/src/main/java/org/keycloak/tests/providers/broker/oidc/TestKeycloakOidcIdentityProviderFactory.java`	Supports max-age broker scenarios.
`tests/custom-providers/src/main/java/org/keycloak/tests/providers/broker/oidc/MissingUseJwksRealmResourceProviderFactory.java`	Creates missing-use JWKS provider.
`tests/custom-providers/src/main/java/org/keycloak/tests/providers/broker/oidc/MissingUseJwksRealmResourceProvider.java`	Serves missing-use JWKS.
`tests/custom-providers/src/main/java/org/keycloak/tests/providers/broker/oidc/ClientIdRequiredJWTClientAuthenticator.java`	Requires JWT client IDs.
`tests/base/src/test/resources/org/keycloak/tests/broker/oidc/vault/consumer_oidc__idp`	Supplies the vault secret.
`tests/base/src/test/java/org/keycloak/tests/broker/saml/KcSamlBrokerTest.java`	Adds baseline JUnit 5 SAML coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/saml/KcSamlSignedDocumentOnlyBrokerTest.java`	Migrates signed-document SAML coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerTest.java`	Adds baseline JUnit 5 OIDC coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerVaultTest.java`	Migrates vault-backed secret coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerUiLocalesWithIdpHintTest.java`	Migrates locale/IdP-hint coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerUiLocalesEnabledTest.java`	Migrates locale-forwarding coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerUiLocalesDisabledTest.java`	Migrates disabled-locale coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerSubMatchIntrospectionTest.java`	Migrates subject-mismatch coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerPromptParameterTest.java`	Migrates prompt forwarding coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerPrivateKeyJwtWithX509HeadersTest.java`	Migrates X.509 JWT coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerPrivateKeyJwtUnsupportedKeyTest.java`	Migrates unsupported-key coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerPrivateKeyJwtTest.java`	Migrates private-key JWT coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerPrivateKeyJwtMissingUseTest.java`	Migrates missing-use JWT coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerPrivateKeyJwtCustomSignAlgTest.java`	Migrates custom-signing coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerPrivateKeyJwtCustomAudienceTest.java`	Migrates custom-audience coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerPrivateKeyJwtClientIdRequiredTest.java`	Migrates required-client-ID coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerPkceTest.java`	Migrates broker PKCE coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerPassMaxAgeTest.java`	Migrates max-age propagation coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerParameterForwardTest.java`	Migrates parameter-forwarding coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerNonceParameterTest.java`	Migrates nonce propagation coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerNoLoginHintTest.java`	Migrates disabled-login-hint coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerIdpHintTest.java`	Migrates IdP-hint scenarios.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerJWEUserInfoJustEncryptedTest.java`	Migrates encrypted-userinfo coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerJWETest.java`	Adds shared JWE behavior tests.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerJWEEcdhEsP384A192GcmTest.java`	Adds a JWE algorithm variant.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerJWEEcdhEsP384A192CbcHs384Test.java`	Adds a JWE algorithm variant.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerJWEEcdhEsA256KwP521A256GcmTest.java`	Adds a JWE algorithm variant.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerJWEEcdhEsA256KwP521A256CbcHs512Test.java`	Adds a JWE algorithm variant.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerJWEEcdhEsA192KwP384A192GcmTest.java`	Adds a JWE algorithm variant.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerJWEEcdhEsA192KwP384A192CbcHs384Test.java`	Adds a JWE algorithm variant.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerJWEEcdhEsA128KwP256A128GcmTest.java`	Adds a JWE algorithm variant.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerJWEEcdhEsA128KwP256A128CbcHs256Test.java`	Adds a JWE algorithm variant.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerIdpPublicKeyMissingUseTest.java`	Migrates IdP JWKS coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerHiddenIdpHintTest.java`	Migrates hidden-IdP coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerColonAliasClientSecretBasicAuthTest.java`	Migrates colon client-ID coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerClientSecretJwtTest.java`	Migrates secret-JWT coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerClientSecretJwtCustomSignAlgTest.java`	Migrates custom-HMAC coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerClientSecretBasicAuthTest.java`	Adds basic-auth broker coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerAcrParameterTest.java`	Migrates ACR forwarding coverage.
`tests/base/src/test/java/org/keycloak/tests/broker/JweBrokerConfigSupport.java`	Configures shared JWE tests.
`tests/base/src/test/java/org/keycloak/tests/broker/EcdhEsJweBrokerConfigSupport.java`	Configures ECDH-ES variants.
`tests/base/src/test/java/org/keycloak/tests/broker/BrokerLoginTest.java`	Provides shared broker login tests.
`test-framework/ui/src/main/java/org/keycloak/testframework/ui/page/LoginTotpPage.java`	Adds an OTP page object.
`test-framework/ui/src/main/java/org/keycloak/testframework/ui/page/IdpConfirmLinkPage.java`	Adds an IdP-link page object.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

vaceksimon

Thanks for putting work into this @sguilhen! Given the scale of this test migration, it's difficult to verify that the migrated tests still serve their original purpose, and the new structure seems harder to read.

A few suggestions to make this easier to review and land:

Break into smaller PRs: Migrating a few test classes (and its underlying superclass logic) per PR has proven much faster to review and safer to land. Even just a single test class per PR is completely fine.
I encourage using the commit-migration script, here's a little demo:

commit-migration.mp4

It helps Git track file moves rather than showing deleted/recreated files, giving us a clean diff to inspect.

Copilot

Pull request overview

Copilot reviewed 95 out of 95 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 100 out of 100 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

tests/base/src/test/java/org/keycloak/tests/broker/saml/KcSamlSignedDocumentOnlyBrokerTest.java:1

The filter compares KeyUse.SIG (an enum) to k.getUse() (typically a String in key metadata representations). As written, KeyUse.SIG.equals(k.getUse()) will never match and will likely trigger orElseThrow(). Compare using the string value instead (e.g., KeyUse.SIG.name().equals(k.getUse())).
tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerPrivateKeyJwtClientIdRequiredTest.java:1
Two issues here can break the test setup: (1) KeyUse.SIG.equals(k.getUse()) likely never matches because k.getUse() is typically a String; compare using KeyUse.SIG.name().equals(k.getUse()). (2) The filter checks only getPublicKey() != null, but the code later uses rsaKey.getCertificate(); if the selected key has no certificate, you'll configure a null certificate. Filter on k.getCertificate() != null (and keep the algorithm/use checks) to ensure the attribute is populated.
tests/base/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerVaultTest.java:1
Using URL#getPath() is brittle across platforms and packaging (URL-encoded characters, Windows drive letters, running from a JAR, etc.). Prefer converting via url.toURI() and resolving a proper filesystem path (or copying resources to a temp dir) before passing it to the server config, to avoid intermittent failures on CI environments.

+        int userCountBefore = consumerRealm.admin().users().count();
+


Add shared test infrastructure (BrokerConfigSupport, KcOidcBrokerConfigSupport, SamlBrokerConfigSupport, BrokerLoginTest) and migrate KcOidcBrokerTest, KcOidcBrokerClientSecretBasicAuthTest, and KcSamlBrokerTest. The shared OIDC config brokers through the Keycloak-specific "keycloak-oidc" identity provider, matching the legacy KcOidcBrokerConfiguration baseline (with prompt=login and loginHint forwarding); a generic-provider variant can be added where a test actually needs it. The provider user is created without a first/last name, as the legacy suite did, so the imported consumer user is incomplete and the consumer's first-broker-login review-profile page is shown. The default helper asserts that page appears before updating it (mirroring the legacy assertCurrent()), and VERIFY_PROFILE is disabled on the provider realm since the new testsuite enables it by default and would otherwise stop the incomplete user at the provider. A tolerant updateAccountInformationIfPresent() variant remains for flows that intentionally skip profile review. The legacy test classes are retained since they carry additional test methods not yet covered by this migration. Closes keycloak#51954 Signed-off-by: Stefan Guilhen <sguilhen@redhat.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…ite oidc package Pure git rename of the eight legacy config-variant and parameter-forwarding broker tests into the new testsuite oidc package. The two phase-1 tests already in the new suite (KcOidcBrokerTest, KcOidcBrokerClientSecretBasicAuthTest) also move from the flat broker package into oidc/. No content changes; the new-testsuite adaptation follows in the next commit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Stefan Guilhen <sguilhen@redhat.com>

Migrate KcOidcBrokerAcrParameterTest, KcOidcBrokerNoLoginHintTest, KcOidcBrokerParameterForwardTest, KcOidcBrokerPkceTest, KcOidcBrokerPromptParameterTest, KcOidcBrokerUiLocalesDisabledTest, KcOidcBrokerUiLocalesEnabledTest, and KcOidcBrokerUiLocalesWithIdpHintTest. These are all config-only variants with no own test methods - they run the two inherited AbstractBrokerTest login tests with different IDP configurations. Adapted for the phase-1 default user, which is created without a first/last name so the first-broker-login review-profile page appears: BrokerConfigSupport disables the VERIFY_PROFILE required action that new-testsuite realms enable by default, and KcOidcBrokerPromptParameterTest clears the IDP-configured prompt so the client's prompt is forwarded, mirroring the legacy configuration. KcOidcBrokerUiLocalesWithIdpHintTest also restores the legacy assertions that the provider login and consumer review-profile pages render in Hungarian. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Stefan Guilhen <sguilhen@redhat.com>

…w testsuite oidc package Pure git rename of the eight legacy client-secret-jwt, private-key-jwt, and sub-mismatch broker tests into the new testsuite oidc package. No content changes; the new-testsuite adaptation follows in the next commit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Stefan Guilhen <sguilhen@redhat.com>

…ests Migrate KcOidcBrokerClientSecretJwtTest, KcOidcBrokerClientSecretJwtCustomSignAlgTest, KcOidcBrokerColonAliasClientSecretBasicAuthTest, KcOidcBrokerPrivateKeyJwtTest, KcOidcBrokerPrivateKeyJwtCustomSignAlgTest, KcOidcBrokerPrivateKeyJwtCustomAudienceTest, KcOidcBrokerPrivateKeyJwtWithX509HeadersTest, and KcOidcBrokerSubMatchIntrospectionTest. Config-only variants except SubMatchIntrospectionTest which overrides testLogInAsUserInIDP and disables loginWithExistingUser - both covered by the new version. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Stefan Guilhen <sguilhen@redhat.com>

Pure git rename of KcOidcBrokerIdpHintTest, KcOidcBrokerHiddenIdpHintTest, and KcOidcBrokerNonceParameterTest into the new testsuite oidc package. No content changes; the new-testsuite adaptation follows in the next commit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Stefan Guilhen <sguilhen@redhat.com>

Migrate KcOidcBrokerIdpHintTest (4 own tests), KcOidcBrokerHiddenIdpHintTest (1 own test), and KcOidcBrokerNonceParameterTest (2 own tests including a new testNonceSet that verifies the nonce is present in the token response). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Stefan Guilhen <sguilhen@redhat.com>

Pure git rename of KcOidcBrokerJWETest, KcOidcBrokerJWEUserInfoJustEncryptedTest, and the eight ECDH-ES variant tests into the new testsuite oidc package. No content changes; the new-testsuite adaptation follows in the next commit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Stefan Guilhen <sguilhen@redhat.com>

Migrate KcOidcBrokerJWETest (1 own test), KcOidcBrokerJWEUserInfoJustEncryptedTest, and all 8 ECDH-ES variant tests. Add shared JweBrokerConfigSupport and EcdhEsJweBrokerConfigSupport interfaces. Also removes AbstractKcOidcBrokerJWEEcdhEsTest since all its subclasses are migrated, and updates the fips-suite file to drop the removed test entries. The shared JweUserInfoBrokerTest interface hosts the encrypted-UserInfo test so every JWE variant runs it, instead of duplicating the method per variant. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Stefan Guilhen <sguilhen@redhat.com>

Pure git rename of the ClientIdRequiredJWTClientAuthenticator, MissingUseJwks, and UnsupportedKeyJwks providers from the legacy testsuite-providers module to tests/custom-providers. No content changes; the package move, SPI updates, and legacy-original removal follow in the next commit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Stefan Guilhen <sguilhen@redhat.com>

…acy originals Port MissingUseJwks, UnsupportedKeyJwks, ClientIdRequiredJWTClientAuthenticator, and TestKeycloakOidcIdentityProviderFactory from the legacy testsuite-providers module to tests/custom-providers for use by the new broker test migrations. The private-key-JWT and IdP-public-key broker tests that used these providers are migrated to the new testsuite, so the moved sources and their SPI registrations are removed from the legacy testsuite-providers module. The two *RestResource classes have no new-testsuite counterpart and are dropped. TestKeycloakOidcIdentityProviderFactory is kept in the legacy module as well because other legacy tests (KcCustomOidcBrokerTest, KcOidcFirstBrokerLoginTest, PasskeysKcOidcFirstBrokerLoginTest, FederatedIdentityModelTest) still use it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Stefan Guilhen <sguilhen@redhat.com>

…kage Pure git rename into the new testsuite oidc package; the new-testsuite adaptation follows in the next commit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Stefan Guilhen <sguilhen@redhat.com>

Uses the MissingUseJwks custom provider to serve JWKS without the "use" field, verifying that broker login still works with public key validation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Stefan Guilhen <sguilhen@redhat.com>

Pure git rename of KcOidcBrokerPrivateKeyJwtMissingUseTest, KcOidcBrokerPrivateKeyJwtUnsupportedKeyTest, and KcOidcBrokerPrivateKeyJwtClientIdRequiredTest into the new testsuite oidc package. No content changes; the new-testsuite adaptation follows in the next commit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Stefan Guilhen <sguilhen@redhat.com>

Migrate KcOidcBrokerPrivateKeyJwtMissingUseTest (JWKS without "use" field), KcOidcBrokerPrivateKeyJwtUnsupportedKeyTest (JWKS with unsupported EC key), and KcOidcBrokerPrivateKeyJwtClientIdRequiredTest (custom client authenticator requiring client_id in request body). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Stefan Guilhen <sguilhen@redhat.com>

Pure git rename into the new testsuite oidc package; the new-testsuite adaptation, vault resource, and legacy config removal follow in the next commit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Stefan Guilhen <sguilhen@redhat.com>

Configures the file-based vault provider and sets the IDP client secret to a vault expression, verifying that broker login works with vault-resolved secrets. Also removes the legacy KcOidcBrokerVaultConfiguration. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Stefan Guilhen <sguilhen@redhat.com>

Pure git rename into the new testsuite oidc package; the new-testsuite adaptation follows in the next commit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Stefan Guilhen <sguilhen@redhat.com>

Tests propagation of the max_age parameter for brokered logins using the TestKeycloakOidcIdentityProviderFactory custom provider. The IDP is created with the test-keycloak-oidc providerId at realm setup time since providerId is immutable after creation via the admin API. The consumer IDP clears the prompt attribute set by the shared config so the broker does not force an interactive login, mirroring the legacy KcOidcBrokerConfigurationWithPassMaxAge; without it the ignore-max-age error path is never exercised. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Stefan Guilhen <sguilhen@redhat.com>

Pure git rename of KcSamlSignedDocumentOnlyBrokerTest and the phase-1 KcSamlBrokerTest from the flat broker package into the new testsuite saml package. No content changes; the new-testsuite adaptation follows in the next commit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Stefan Guilhen <sguilhen@redhat.com>

Configures SAML provider to sign documents only (not assertions) and the consumer IDP to validate signatures but not require signed assertions. The legacy test had a bug where KcSamlSignedBrokerConfiguration inherited its INSTANCE from KcSamlBrokerConfiguration, so the signing config overrides were never applied. The new test correctly applies the configuration via @beforeeach using the provider realm's auto-generated RSA signing certificate. The migration also restores the legacy signed-AuthnRequest coverage that the first migration dropped: client signature validation on the provider SAML client (saml.client.signature plus the consumer's signing certificate) and wantAuthnRequestsSigned on the consumer IdP. Closes keycloak#51955 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Stefan Guilhen <sguilhen@redhat.com>

Copilot

Pull request overview

Copilot reviewed 100 out of 100 changed files in this pull request and generated 1 comment.

+        String samlClientId = "http://localhost:8080/realms/" + CONSUMER_REALM;
+        ClientRepresentation client = providerRealm.admin().clients().findByClientId(samlClientId).get(0);


sguilhen · 2026-08-26T12:59:05Z

Thanks for putting work into this @sguilhen! Given the scale of this test migration, it's difficult to verify that the migrated tests still serve their original purpose, and the new structure seems harder to read.

A few suggestions to make this easier to review and land:

Break into smaller PRs: Migrating a few test classes (and its underlying superclass logic) per PR has proven much faster to review and safer to land. Even just a single test class per PR is completely fine.

I encourage using the commit-migration script, here's a little demo:

commit-migration.mp4
It helps Git track file moves rather than showing deleted/recreated files, giving us a clean diff to inspect.

@vaceksimon Thanks for the input. I'll rework this to use the script if it makes it easier to review. I'll try to break it into smaller PRs, but one PR per class is impractical to do. I'll just hold until the PR that lays the baseline (#51859) is merged as all changes there end up cascading through this one. And if I break into smaller PRs now I will need to rebase all of them on any change happening there.

One thing to note though: even after reducing the PR size into smaller chunks, there will be quite some changes - the broker tests have a complex structure that is being migrated into the new testsuite, where things like events, custom providers, etc, have their own way of being used. So the changes can be quite significant even after using the "move" script.

sguilhen · 2026-09-03T13:41:12Z

Closing this in favor of smaller/more focused PRs.

sguilhen changed the title ~~Broker test migration phase 2~~ Migrate OIDC/SAML broker config-variant and infrastructure-dependent tests to the new testsuite Aug 21, 2026

sguilhen force-pushed the broker-test-migration-phase-2 branch 2 times, most recently from 9b53b6e to ea31c21 Compare August 21, 2026 21:50

sguilhen mentioned this pull request Aug 22, 2026

Migrate config-variant and infrastructure-dependent broker tests #51955

Closed

sguilhen force-pushed the broker-test-migration-phase-2 branch 2 times, most recently from c530ef9 to f2620ae Compare August 23, 2026 16:50

sguilhen marked this pull request as ready for review August 24, 2026 12:17

Copilot AI balanced review requested due to automatic review settings August 24, 2026 12:17

sguilhen requested review from a team as code owners August 24, 2026 12:17

Copilot started reviewing on behalf of sguilhen August 24, 2026 12:17

View session

Copilot AI reviewed Aug 24, 2026

View reviewed changes

martin-kanis requested a review from vaceksimon August 25, 2026 09:51

vaceksimon reviewed Aug 25, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings August 25, 2026 12:44

sguilhen force-pushed the broker-test-migration-phase-2 branch from f2620ae to 9285823 Compare August 25, 2026 12:44

Copilot started reviewing on behalf of sguilhen August 25, 2026 12:57

View session

Copilot AI reviewed Aug 25, 2026

View reviewed changes

Comment thread tests/base/src/test/java/org/keycloak/tests/broker/EcdhEsJweBrokerConfigSupport.java Resolved

Comment thread ...e/src/test/java/org/keycloak/tests/broker/oidc/KcOidcBrokerJWEUserInfoJustEncryptedTest.java Resolved

keycloak-github-bot Bot added the flaky-test label Aug 25, 2026

keycloak-github-bot Bot mentioned this pull request Aug 25, 2026

Flaky test: org.keycloak.testsuite.oauth.AccessTokenTest#accessTokenRequest #51102

Closed

Copilot AI review requested due to automatic review settings August 25, 2026 16:52

sguilhen force-pushed the broker-test-migration-phase-2 branch from 9285823 to 5706cd3 Compare August 25, 2026 16:52

Copilot AI reviewed Aug 25, 2026

View reviewed changes

Comment thread tests/base/src/test/java/org/keycloak/tests/broker/BrokerLoginTest.java

Comment on lines +46 to +47

int userCountBefore = consumerRealm.admin().users().count();

Copilot started reviewing on behalf of sguilhen August 25, 2026 18:24

View session

sguilhen and others added 4 commits August 25, 2026 15:56

sguilhen and others added 17 commits August 25, 2026 16:21

Move KcOidcBrokerPassMaxAgeTest to new testsuite oidc package

aef26cd

Pure git rename into the new testsuite oidc package; the new-testsuite adaptation follows in the next commit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Stefan Guilhen <sguilhen@redhat.com>

Copilot AI review requested due to automatic review settings August 26, 2026 09:53

sguilhen force-pushed the broker-test-migration-phase-2 branch from 5706cd3 to 30f8dea Compare August 26, 2026 09:53

Copilot started reviewing on behalf of sguilhen August 26, 2026 09:53

View session

Copilot AI reviewed Aug 26, 2026

View reviewed changes

stianst added the team/core-iam label Sep 3, 2026

sguilhen closed this Sep 3, 2026

sguilhen deleted the broker-test-migration-phase-2 branch September 21, 2026 18:58

Navigation Menu

Repository navigation

Migrate OIDC/SAML broker config-variant and infrastructure-dependent tests to the new testsuite - #51949

Migrate OIDC/SAML broker config-variant and infrastructure-dependent tests to the new testsuite#51949
sguilhen wants to merge 21 commits into
keycloak:mainfrom
sguilhen:broker-test-migration-phase-2

sguilhen commented Aug 21, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vaceksimon left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

sguilhen commented Aug 26, 2026 •

edited

Loading

Uh oh!

sguilhen commented Sep 3, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		int userCountBefore = consumerRealm.admin().users().count();

		String samlClientId = "http://localhost:8080/realms/" + CONSUMER_REALM;
		ClientRepresentation client = providerRealm.admin().clients().findByClientId(samlClientId).get(0);

Conversation

sguilhen commented Aug 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What's included

Testing

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vaceksimon left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

sguilhen commented Aug 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sguilhen commented Sep 3, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sguilhen commented Aug 21, 2026 •

edited

Loading

sguilhen commented Aug 26, 2026 •

edited

Loading