Payara 6 on RH JDK in FIPS mode

Technical Discussion
1

Is it possible to run payara 6.2025.1 server on RHEL 9’s openJDK 21 in FIPS mode? When attempted, asadmin create-domain fails with

CLI130: Could not create domain, spdf
Command create-domain failed.
ERROR: create-domain failed.

and no other error message. domain1 will start but and attempt to log into admin (4848) results in the following in the log

[2025-03-13T09:36:59.227-0400] [Payara 6.2025.1] [WARNING] [] [org.glassfish.jersey.internal.Errors] [tid: _ThreadID=285 _ThreadName=Thread-28] [timeMillis: 1741873019227] [levelValue: 900] [[
  The following warnings have been detected: WARNING: Unknown HK2 failure detected:
MultiException stack 1 of 2
java.lang.RuntimeException: java.lang.RuntimeException: java.security.KeyStoreException: JCEKS not found
        at org.glassfish.security.services.impl.JCEKSPasswordAliasStore.containsKey(JCEKSPasswordAliasStore.java:169)
        at fish.payara.nucleus.microprofile.config.source.PasswordAliasConfigSource.getValue(PasswordAliasConfigSource.java:101)
        at fish.payara.nucleus.microprofile.config.spi.ConfigExpressionResolver.getValue(ConfigExpressionResolver.java:135)
        at fish.payara.nucleus.microprofile.config.spi.ConfigExpressionResolver.resolve(ConfigExpressionResolver.java:108)
...

Things will work if JDK_JAVA_OPTIONS='-Dcom.redhat.fips=false' but I was wondering if it is possible to run with FIPS enabled. Any alternatives would be appreciated.

2

Hi @bernie,

Payara Server can be run in FIPS compatible mode on RHEL but to do so you have to install and configure a FIPS-supported JDK security provider like BouncyCastle (see here for more information: https://downloads.bouncycastle.org/fips-java/docs/BC-FJA-UserGuide-2.0.0.pdf) in the JDK running in the server for the Payara Server domain to properly start.

3

We are using RH’s OpenJDK (from the official RHEL repository) which is FIPS compatible. Are you confirming that payara cannot be configured to run on RH’s OpenJDK without disabling the FIPS mode?

4

Greetings @bernie,

No, Payara Server is compatible with FIPS mode in RHEL, but you are still responsible for installing a FIPS-compatible security provider in the JDK. Redhat’s flavor of the OpenJDK has a FIPS compatible security provider (NSS), but it is not installed by default. Once the provider is installed, you’ll have to configure the JDK to use the provider and reconfigure Payara Server to use this provider as well.

5

From the link you provided,

At startup, Red Hat build of OpenJDK 17 checks if the system FIPS policy is enabled. If this policy is enabled, Red Hat build of OpenJDK 17 performs a series of automatic configurations that are intended to help Java applications to comply with FIPS requirements.

These automatic configurations include the following actions:

  • Installing a restricted list of security providers that contains the FIPS-certified Network Security Services (NSS) software token module for cryptographic operations
  • Enforcing the Red Hat Enterprise Linux (RHEL) FIPS crypto-policy for Java that limits the algorithms and parameters available

We have the system FIPS policy enabled (OS level) and according to RH’s documentation, “Red Hat build of OpenJDK performs a series of automatic configurations…” And when I attempt to run payara, it fails. But if I set JDK_JAVA_OPTIONS='-Dcom.redhat.fips=false', payara starts. It still looks to me has if payara will not run on RH’s OpenJDK when it is run in fips mode.