I have a problem with Payara secure admin console. Payara secure admin console suddenly stopped working, but nothing has changed in Payara’s configuration for months.
Payara’s version is 5.2022.3 CE (same thing happens with 5.2022.5), and Java is OpenJDK 11.0.22. Version 5 had to be used because of the legacy app involved. A new domain has been created instead of “domain1”.
First, the server log shows the following error:
[2024-04-04T10:20:09.257+0200] [Payara 5.2022.3] [SEVERE] [] [org.glassfish.admingui] [tid: _ThreadID=214 _ThreadName=admin-thread-pool::admin-listener(8)] [timeMillis: 1712218809257] [levelValue: 1000] [[
javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target;
javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target;
restRequest: endpoint=https://localhost:4848/management/domain/anonymous-user-enabled
attrs={}
method=GET]]
Then follows:
[2024-04-04T10:20:09.628+0200] [Payara 5.2022.3] [SEVERE] [] [] [tid: _ThreadID=213 _ThreadName=admin-thread-pool::admin-listener(7)] [timeMillis: 1712218809628] [levelValue: 1000] [[
javax.net.ssl|ERROR|D5|admin-thread-pool::admin-listener(7)|2024-04-04 10:20:09.627 CEST|TransportContext.java:352|Fatal (CERTIFICATE_UNKNOWN): PKIX path building failed: sun.security.provider.certpath.SunCert
PathBuilderException: unable to find valid certification path to requested target (
"throwable" : {
sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:439)
at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306)
at java.base/sun.security.validator.Validator.validate(Validator.java:264)
at java.base/sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:313)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:222)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:129)
at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:638)
at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:473)
at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:369)
at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443)
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:421)
at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:183)
at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:172)
at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1511)
at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1421)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:456)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:427)
at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:580)
at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:201)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1592)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1520)
at java.base/java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:527)
at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:334)
at org.glassfish.jersey.client.internal.HttpUrlConnector._apply(HttpUrlConnector.java:377)
at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:266)
at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:297)
at org.glassfish.jersey.client.JerseyInvocation.lambda$invoke$1(JerseyInvocation.java:675)
at org.glassfish.jersey.client.JerseyInvocation.call(JerseyInvocation.java:697)
at org.glassfish.jersey.client.JerseyInvocation.lambda$runInScope$3(JerseyInvocation.java:691)
at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
at org.glassfish.jersey.internal.Errors.process(Errors.java:205)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:390)
at org.glassfish.jersey.client.JerseyInvocation.runInScope(JerseyInvocation.java:691)
at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:674)
at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:422)
at org.glassfish.jersey.client.JerseyInvocation$Builder.get(JerseyInvocation.java:318)
at org.glassfish.admingui.common.util.RestUtil.get(RestUtil.java:874)
at org.glassfish.admingui.common.util.RestUtil.restRequest(RestUtil.java:216)
at org.glassfish.admingui.common.handlers.RestApiHandlers.restRequest(RestApiHandlers.java:230)
at jdk.internal.reflect.GeneratedMethodAccessor336.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.sun.jsftemplating.layout.descriptors.handler.Handler.invoke(Handler.java:436)
at com.sun.jsftemplating.layout.descriptors.LayoutElementBase.dispatchHandlers(LayoutElementBase.java:414)
at com.sun.jsftemplating.layout.descriptors.LayoutElementBase.dispatchHandlers(LayoutElementBase.java:388)
at com.sun.jsftemplating.layout.descriptors.LayoutComponent.beforeCreate(LayoutComponent.java:342)
at com.sun.jsftemplating.layout.descriptors.LayoutComponent.getChild(LayoutComponent.java:282)
at com.sun.jsftemplating.layout.LayoutViewHandler.buildUIComponentTree(LayoutViewHandler.java:551)
at com.sun.jsftemplating.layout.LayoutViewHandler.createView(LayoutViewHandler.java:249)
at com.sun.faces.lifecycle.RestoreViewPhase.execute(RestoreViewPhase.java:238)
at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:76)
at com.sun.faces.lifecycle.RestoreViewPhase.doPhase(RestoreViewPhase.java:110)
at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:177)
at javax.faces.webapp.FacesServlet.executeLifecyle(FacesServlet.java:707)
at javax.faces.webapp.FacesServlet.service(FacesServlet.java:451)
at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1637)
at org.apache.catalina.core.ApplicationDispatcher.doInvoke(ApplicationDispatcher.java:818)
at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:683)
at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:527)
at org.apache.catalina.core.ApplicationDispatcher.doDispatch(ApplicationDispatcher.java:497)
at org.apache.catalina.core.ApplicationDispatcher.dispatch(ApplicationDispatcher.java:379)
at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:329)
at org.glassfish.admingui.common.security.AdminConsoleAuthModule.saveRequestAndForwardToLogin(AdminConsoleAuthModule.java:334)
at org.glassfish.admingui.common.security.AdminConsoleAuthModule.validateRequest(AdminConsoleAuthModule.java:197)
at com.sun.enterprise.security.jaspic.config.GFServerConfigProvider$GFServerAuthContext.validateRequest(GFServerConfigProvider.java:867)
at com.sun.web.security.realmadapter.JaspicRealm.validateRequest(JaspicRealm.java:391)
at com.sun.web.security.realmadapter.JaspicRealm.validateRequest(JaspicRealm.java:358)
at com.sun.web.security.realmadapter.JaspicRealm.validateRequest(JaspicRealm.java:181)
at com.sun.web.security.RealmAdapter.invokeAuthenticateDelegate(RealmAdapter.java:487)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:468)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:726)
at org.apache.catalina.core.StandardPipeline.doChainInvoke(StandardPipeline.java:581)
at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:97)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:158)
at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:372)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:239)
at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:520)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:217)
at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:182)
at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:156)
at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:201)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:95)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53)
at org.glassfish.grizzly.portunif.PUFilter.handleRead(PUFilter.java:208)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:95)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53)
at org.glassfish.grizzly.portunif.PUFilter.handleRead(PUFilter.java:208)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:95)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:524)
at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:89)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:94)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:33)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:114)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:569)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:549)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:148)
at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:129)
at java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297)
at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:434)
... 110 more}
And after that, it’s not possible to login to Payara’s secure admin console. “j_security_check” page comes blank. After the domain is restarted, the secure admin console login works for a few minutes until the above mentioned errors appear again.
s1as and glassfish-instance certificates are valid and the same in both keystores (cacerts.jks and keystore.jks). As I mentioned at the beginning, nothing has been changed or altered in Payara’s configuration for months.
I’m aware that this version of Payara CE is not supported, but maybe someone has an idea what might be the cause of this problem?