I have a basic configuration

Browser –> Spring BFF –> Spring Rest API

The Spring BFF and Spring Rest API are both housed in AWS (in Docker containers running on ECS)

Now, I had problems initially with the Spring Rest API not being able to get a JwTKey (from an auth0 endpoint)
I later found out because it was not preferring Ipv6.

So I added an IPV6 Kotlin Object, and used it here:

Spring Rest API

/**
 * Configuration class for JWT handling with IPv6 support.
 * Provides JWT decoder configuration with custom IPv6 resolver and caching mechanisms.
 *
 * @property serverProperties Server properties containing Auth0 JWK set URI configuration
 */
@Configuration
internal class JwtConfig(
    private val serverProperties: ServerProperties,
) {

    companion object {
        private val logger = LoggerFactory.getLogger(JwtConfig::class.java)
    }

    /**
     * Custom implementation of JWKSetSource for retrieving and caching JSON Web Key Sets.
     *
     * @property webClient WebClient configured for IPv6 connections
     * @property jwkSetUri URI endpoint for retrieving the JWK set
     */
    private inner class CustomJwkSetSource(
        private val webClient: WebClient,
        private val jwkSetUri: URI
    ) : JWKSetSource<SecurityContext> {
        private var cachedJwkSet: JWKSet? = null

        /**
         * Retrieves the JWK set from the configured endpoint with caching support.
         * Attempts to fetch a fresh JWK set and falls back to cached version if available.
         *
         * @param refreshEvaluator Optional evaluator for cache refresh decisions
         * @param currentTime Current timestamp for cache evaluation
         * @param context Optional security context for the retrieval operation
         * @return JWKSet containing the retrieved or cached key set
         * @throws JwkSetRetrievalException if retrieval fails and no cached version is available
         */
        override fun getJWKSet(
            refreshEvaluator: JWKSetCacheRefreshEvaluator?,
            currentTime: Long,
            context: SecurityContext?
        ): JWKSet {
            try {
                val response = webClient.get()
                    .uri(jwkSetUri)
                    .retrieve()
                    .toEntity(String::class.java)
                    .block(Duration.ofSeconds(30))
                    ?: return handleJwkSetError("Failed to retrieve JWK set - null response")

                val body = response.body
                    ?: return handleJwkSetError("Empty JWK set response")

                return try {
                    JWKSet.parse(body).also {
                        cachedJwkSet = it
                        logger.debug("Successfully retrieved and cached new JWK set")
                    }
                } catch (e: Exception) {
                    logger.error("Failed to parse JWK set response", e)
                    handleJwkSetError("Invalid JWK set format: ${e.message}")
                }
            } catch (ex: Exception) {
                logger.error("Error retrieving JWK set", ex)
                return handleJwkSetError("Failed to retrieve JWK set: ${ex.message}")
            }
        }

        /**
         * Handles JWK set retrieval errors by attempting to use cached data.
         * Logs error details and attempts to provide a cached version if available.
         *
         * @param errorMessage Detailed description of the error that occurred
         * @return Cached JWKSet if available
         * @throws JwkSetRetrievalException if no cached data is available
         */
        private fun handleJwkSetError(errorMessage: String): JWKSet {
            cachedJwkSet?.let {
                logger.error("$errorMessage. Falling back to cached version")
                return it
            }

            logger.error("$errorMessage. No cached version available")
            throw JwkSetRetrievalException(errorMessage)
        }

        override fun close() {
            // No resources to clean up
        }
    }

    /**
     * Exception thrown when JWK set retrieval fails and no cached version is available.
     *
     * @param message Detailed error message describing the failure
     */
    private class JwkSetRetrievalException(message: String) : RuntimeException(message)

    /**
     * Creates and configures a ReactiveJwtDecoder bean with IPv6 support.
     * Configures the decoder with:
     * - Custom IPv6 resolver
     * - JWK set caching
     * - Rate limiting
     * - Error handling
     *
     * @return Configured ReactiveJwtDecoder instance
     */
    @Bean
    fun jwtDecoder(): ReactiveJwtDecoder {
        val jwkSetUri = URI(serverProperties.auth0JwKeySetUri)
        val host = jwkSetUri.host

        logger.debug("Initializing JWT decoder for host: $host")

        val jwkSource = JWKSourceBuilder.create(CustomJwkSetSource(createIpv6WebClient(host), jwkSetUri))
            .cache(
                TimeUnit.HOURS.toMillis(12),
                TimeUnit.MINUTES.toMillis(60)
            )
            .rateLimited(TimeUnit.MINUTES.toMillis(10)) { event ->
                logger.warn("Rate limit reached for JWK source")
            }
            .refreshAheadCache(true)
            .build()

        val jwtProcessor = DefaultJWTProcessor<SecurityContext>().apply {
            setJWSKeySelector(JWSAlgorithmFamilyJWSKeySelector.fromJWKSource(jwkSource))
        }

        return NimbusReactiveJwtDecoder { jwt ->
            Mono.fromCallable {
                try {
                    jwtProcessor.process(jwt, null)
                } catch (ex: Exception) {
                    logger.error("JWT processing error", ex)
                    throw ex
                }
            }
        }
    }

}

IPV6 kotlin object

**
 * Utility object for IPv6-enabled WebClient creation and DNS resolution.
 * Provides reusable components for IPv6 network connectivity with IPv4 fallback.
 */
internal object Ipv6WebClientUtil {

    private val logger = LoggerFactory.getLogger(Ipv6WebClientUtil::class.java)

    /**
     * Initialize IPv6 preferences when the object is first accessed.
     * These are JVM-wide settings that should be set once at startup.
     */
    init {
        initializeIpv6Preferences()
    }

    /**
     * Initializes IPv6 preferences for the JVM and logs current settings.
     * This should only be called once during application startup.
     */
    private fun initializeIpv6Preferences() {
        // Set IPv6 preferences
        System.setProperty("java.net.preferIPv6Addresses", "true")
        System.setProperty("java.net.preferIPv6Stack", "true")
        java.security.Security.setProperty("networkaddress.preferIPv6Addresses", "true")

        // Log the current settings
        logger.debug("IPv6 Preferences:")
        logger.debug("preferIPv6Addresses: ${System.getProperty("java.net.preferIPv6Addresses")}")
        logger.debug("preferIPv6Stack: ${System.getProperty("java.net.preferIPv6Stack")}")
        logger.debug("Security preferIPv6Addresses: ${java.security.Security.getProperty("networkaddress.preferIPv6Addresses")}")
    }

    /**
     * Creates a custom IPv6 address resolver for Netty.
     * Handles both single and multiple address resolution with IPv6 preference.
     * Falls back to IPv4 if IPv6 is not available
     *
     * @param host The hostname to resolve
     * @return AddressResolverGroup configured for IPv6 resolution
     */
    fun createIpv6Resolver(host: String): AddressResolverGroup<InetSocketAddress> {
        return object : AddressResolverGroup<InetSocketAddress>() {
            override fun newResolver(executor: EventExecutor) =
                object : AbstractAddressResolver<InetSocketAddress>(executor) {
                    override fun doResolve(address: InetSocketAddress?, promise: Promise<InetSocketAddress>?) {
                        try {
                            val hostname = address?.hostName ?: host
                            val port = address?.port ?: 443

                            logger.debug("Attempting to resolve IPv6 address for: $hostname:$port")

                            // Try IPv6 first, then fall back to IPv4
                            val resolvedAddress = InetAddress.getAllByName(hostname)
                                .firstOrNull { it is Inet6Address }
                                ?.let { InetSocketAddress(it, port) }
                                ?: InetAddress.getAllByName(hostname)
                                    .firstOrNull()
                                    ?.let { InetSocketAddress(it, port) }
                                ?: throw IllegalStateException("No address available for $hostname")

                            logger.debug("Resolved address: ${resolvedAddress.address.hostAddress}:$port")
                            promise?.setSuccess(resolvedAddress)
                        } catch (ex: Exception) {
                            logger.error("Failed to resolve IPv6 address", ex)
                            promise?.setFailure(ex)
                        }
                    }

                    override fun doResolveAll(address: InetSocketAddress?, promise: Promise<MutableList<InetSocketAddress>>?) {
                        try {
                            val hostname = address?.hostName ?: host
                            val port = address?.port ?: 443

                            val addresses = InetAddress.getAllByName(hostname)
                                .map { InetSocketAddress(it, port) }
                                .sortedBy { it.address !is Inet6Address }
                                .toMutableList()

                            if (addresses.isEmpty()) {
                                promise?.setFailure(IllegalStateException("No addresses available for $hostname"))
                            } else {
                                promise?.setSuccess(addresses)
                            }
                        } catch (e: Exception) {
                            logger.error("Failed to resolve addresses", e)
                            promise?.setFailure(e)
                        }
                    }

                    override fun doIsResolved(address: InetSocketAddress?) =
                        address?.address != null && !address.isUnresolved
                }
        }
    }

    /**
     * Creates an IPv6-enabled WebClient for token endpoint communications.
     *
     * Features:
     * - IPv6 address resolution with IPv4 fallback
     * - Custom timeout configuration
     * - Secure HTTPS support
     * - Connection logging
     *
     * @param host The token endpoint hostname to resolve
     * @return WebClient configured with IPv6 support
     */
    fun createIpv6WebClient(host: String): WebClient {
        return WebClient.builder()
            .clientConnector(
                ReactorClientHttpConnector(
                    HttpClient.from(
                        TcpClient.create()
                            .resolver(createIpv6Resolver(host))
                            .option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 30000)
                    ).secure()
                )
            )
            .build()
    }

}

Then I had a problem in the BFF, in that after successful authorization with Auth0, and successful de-serialisation of the Authorization Request object, It just paused there, and would not initiate token exchange with Auth0.

AWS

Compared to local host

So again, using the same kotlin object in my rest api, I used it and modified the following BFF class

See the Client Configuration Provider and Create Refresh Provider funs, where you should see the createIpv6 client

// adapted from:
// https://github.com/ch4mpy/spring-addons/blob/master/spring-addons-starter-oidc/src/main/java/com/c4_soft/springaddons/security/oidc/starter/reactive/client/PerRegistrationReactiveOAuth2AuthorizedClientProvider.java

/**
 * Custom OAuth2 Authorized Client Provider that manages providers per client registration.
 *
 * Unlike the default DelegatingReactiveOAuth2AuthorizedClientProvider, this implementation:
 * 1. Maintains separate providers for each OAuth2 client registration
 * 2. Allows custom parameters for token requests per client
 * 3. Supports different grant types (Authorization Code, Client Credentials)
 *
 * Token Request Customization:
 * - Supports extra parameters for token requests
 * - Handles refresh token scenarios
 * - Configures client-specific token response clients
 *
 * Grant Types Supported:
 * - Authorization Code (with optional refresh token support)
 * - Client Credentials
 *
 * @property clientRegistrationRepository Repository of OAuth2 client registrations
 * @property requestParameterProperties Configuration properties for token request parameters
 * @property customProvidersByRegistrationId Map of custom providers per registration ID
 * @since 1.0.0
 */
internal class PerRegistrationReactiveOAuth2AuthorizedClientProvider(
    clientRegistrationRepository: ReactiveClientRegistrationRepository,
    private val requestParameterProperties: RequestParameterProperties,
    private val customProvidersByRegistrationId: Map<String, List<ReactiveOAuth2AuthorizedClientProvider>>,

    ) : ReactiveOAuth2AuthorizedClientProvider {

    companion object {
        private val logger = LoggerFactory.getLogger(PerRegistrationReactiveOAuth2AuthorizedClientProvider::class.java)
        private const val OFFLINE_ACCESS_SCOPE = "offline_access"
    }

    private val providersByRegistrationId = ConcurrentHashMap<String, DelegatingReactiveOAuth2AuthorizedClientProvider>()

    init {
        initializeProviders(clientRegistrationRepository)
    }

    /**
     * Initializes providers for each client registration.
     *
     * Creates and stores a delegating provider for each registration in the repository.
     * Each provider is configured based on the registration's grant type and requirements.
     *
     * @param clientRegistrationRepository Source of OAuth2 client registrations
     */
    private fun initializeProviders(clientRegistrationRepository: ReactiveClientRegistrationRepository) {
        (clientRegistrationRepository as? InMemoryReactiveClientRegistrationRepository)?.toList()?.forEach { registration ->
            val delegate = createDelegatingProvider(registration)
            logger.debug(
                "Initialized provider - Registration ID: {}, Client Name: {}",
                registration.registrationId,
                registration.clientName
            )
            providersByRegistrationId[registration.registrationId] = delegate
        }
    }

    /**
     * Creates delegating provider for specific client registration.
     *
     * Configures a provider with appropriate sub-providers based on the
     * registration's grant type and configuration.
     *
     * @param registration Client registration to create provider for
     * @return Configured delegating provider
     */
    private fun createDelegatingProvider(
        registration: ClientRegistration
    ): DelegatingReactiveOAuth2AuthorizedClientProvider =
        DelegatingReactiveOAuth2AuthorizedClientProvider(
            getProvidersFor(registration, requestParameterProperties)
        )

    /**
     * Authorizes OAuth2 clients using registration-specific providers.
     *
     * Uses a cached provider for the registration if available, or creates
     * a new one if needed. The provider handles the specific authorization
     * flow based on the registration's configuration.
     *
     * @param context Authorization context containing client registration and details
     * @return Mono containing authorized client if successful
     */
    override fun authorize(context: OAuth2AuthorizationContext?): Mono<OAuth2AuthorizedClient> {
        context ?: return Mono.empty()

        val registration = context.clientRegistration

        // Initialize provider if not exists
        if (!providersByRegistrationId.containsKey(registration.registrationId)) {
            providersByRegistrationId[registration.registrationId] = createDelegatingProvider(registration)
        }

        return providersByRegistrationId[registration.registrationId]!!.authorize(context)
    }

    /**
     * Determines and configures appropriate providers based on grant type.
     *
     * Supports:
     * - Authorization Code grant with optional refresh token
     * - Client Credentials grant
     * - Custom providers specified in configuration
     *
     * @param registration Client registration to configure providers for
     * @param requestParameterProperties Properties for customizing token requests
     * @return List of configured providers for the registration
     */
    private fun getProvidersFor(
        registration: ClientRegistration,
        requestParameterProperties: RequestParameterProperties
    ): List<ReactiveOAuth2AuthorizedClientProvider> {

        // get providers for the given client registration id (as passed in from the class constructor)
        val providers = ArrayList(
            customProvidersByRegistrationId[registration.registrationId] ?: listOf()
        )

        when (registration.authorizationGrantType) {

            // if grant type is authorisation code, add authorization code provider
            // also add refresh token provider (if 'offline_access' scope is provided)
            AuthorizationGrantType.AUTHORIZATION_CODE -> {

                // add basic auth code provider - it just triggers the auth flow
                providers.add(AuthorizationCodeReactiveOAuth2AuthorizedClientProvider())

                // add refresh token provider if scope includes offline access
                if (registration.scopes.contains(OFFLINE_ACCESS_SCOPE)) {
                    providers.add(createRefreshTokenProvider(registration, requestParameterProperties))
                }
            }

            // if grant type is client credentials, add client credentials provider
            AuthorizationGrantType.CLIENT_CREDENTIALS -> {
                providers.add(createClientCredentialsProvider(registration, requestParameterProperties))
            }
        }
        return providers
    }


    /**
     * Creates client credentials provider with optional extra parameters.
     *
     * Configures a provider that supports additional token request parameters
     * specified in the registration's configuration.
     *
     * @param registration Client registration to configure provider for
     * @param requestParameterProperties Source of extra token parameters
     * @return Configured client credentials provider
     */
    private fun createClientCredentialsProvider(
        registration: ClientRegistration,
        requestParameterProperties: RequestParameterProperties
    ): ClientCredentialsReactiveOAuth2AuthorizedClientProvider {

        // create provider and get extraParameters
        val provider = ClientCredentialsReactiveOAuth2AuthorizedClientProvider()
        val extraParams = requestParameterProperties.getExtraTokenParameters(registration.registrationId)
            .takeIf { it.isNotEmpty() } ?: return provider

        // get token endpoint host for IPv6 resolution
        val tokenEndpoint = URI(registration.providerDetails.tokenUri)

        // create response client with IPv6 support and extra parameters
        return provider.apply {
            setAccessTokenResponseClient(
                WebClientReactiveClientCredentialsTokenResponseClient().apply {
                    setWebClient(createIpv6WebClient(tokenEndpoint.host))
                    addParametersConverter { extraParams }
                }
            )
        }
    }

    /**
     * Creates refresh token provider with optional extra parameters.
     *
     * Configures a provider that supports token refresh with additional
     * parameters specified in the registration's configuration.
     *
     * @param registration Client registration to configure provider for
     * @param requestParameterProperties Source of extra token parameters
     * @return Configured refresh token provider
     */
    private fun createRefreshTokenProvider(
        registration: ClientRegistration,
        requestParameterProperties: RequestParameterProperties
    ): RefreshTokenReactiveOAuth2AuthorizedClientProvider {

        // create provider and get extraParameters
        val provider = RefreshTokenReactiveOAuth2AuthorizedClientProvider()
        val extraParams = requestParameterProperties.getExtraTokenParameters(registration.registrationId)
            .takeIf { it.isNotEmpty() } ?: return provider

        // get token endpoint host for IPv6 resolution
        val tokenEndpoint = URI(registration.providerDetails.tokenUri)

        // create response client with IPv6 support and extra parameters
        return provider.apply {
            setAccessTokenResponseClient(
                WebClientReactiveRefreshTokenTokenResponseClient().apply {
                    setWebClient(createIpv6WebClient(tokenEndpoint.host))
                    addParametersConverter { extraParams }
                }
            )
        }
    }

}

I left this provider unchanged, as it didn’t have a setAccessTokenResponseClient method
providers.add(AuthorizationCodeReactiveOAuth2AuthorizedClientProvider())

My OAuthroizationManagerConfig class is unchanged

@Configuration
internal class OAuth2AuthorizedManagerConfig {

    companion object {
        private val logger = LoggerFactory.getLogger(OAuth2AuthorizedManagerConfig::class.java)
    }

    /**
     * Creates and configures the ReactiveOAuth2AuthorizedClientManager.
     *
     * Manager responsibilities:
     * - Coordinates client registration access
     * - Manages token lifecycle
     * - Handles authorization persistence
     * - Provides reactive processing
     *
     * Security measures:
     * - Secure component integration
     * - Token handling procedures
     * - State validation
     * - Operation logging
     *
     * @param reactiveClientRegistrationRepository Source of OAuth2 client registrations
     * @param redisServerOAuth2AuthorizedClientRepository Persistence for authorized clients
     * @param reactiveAuthorizedClientProvider Token lifecycle manager
     * @return Configured [ReactiveOAuth2AuthorizedClientManager]
     */
    @Bean
    fun reactiveAuthorizedClientManager(
        reactiveClientRegistrationRepository: ReactiveClientRegistrationRepository,
        redisServerOAuth2AuthorizedClientRepository: RedisServerOAuth2AuthorizedClientRepository,
        reactiveAuthorizedClientProvider: ReactiveOAuth2AuthorizedClientProvider,
    ): ReactiveOAuth2AuthorizedClientManager = DefaultReactiveOAuth2AuthorizedClientManager(
        reactiveClientRegistrationRepository,
        redisServerOAuth2AuthorizedClientRepository
    ).apply {
        logger.debug("Configuring OAuth2 authorized client manager")
        setAuthorizedClientProvider(reactiveAuthorizedClientProvider)
    }

}

But despite making the above changes, the BFF still just pauses from what I can see on the AWS CloudWatch logs, just before Token Exchange, which I think is what the AuthorizationCodeReactiveOAuth2AuthorizedClientProvider is for.

I’m not sure if it is an Ipv6 issue, as I later added a Nat gateway allowing for Ipv4 access, but that didn’t change anything.

I’m not sure if anyone can help, or know what the issue actually is.

On my local host, everything works fine.

My Auth0 logs show ‘succesful exchange’ when logging in from local host, but only ‘successful login’ when doing this from AWS.