Periodic connection closed exception in server.log

Technical Discussion
1

Hello,

This is not a new issue, but the bits of the trace have changed in various releases. I suspect this is just a client disconnecting while using a longer-running endpoint (such as video playback, or file downloading), and I’m wondering how to suppress these particular exceptions, or if it’s a bug in catalina or grizzly. I have seen code in Glassfish which suppresses ClientAbortException, and maybe that code just isn’t as up to date with the newer types of listeners and connectors. We see this 1-3 times a day on average. OS is Windows, JDK is Oracle 21, Payara version is 6.2023.12.

[2024-05-30T14:57:54.542-0400] [Payara 6.2023.12] [WARNING] [] [jakarta.enterprise.web] [tid: _ThreadID=231 _ThreadName=http-thread-pool::http-listener-2(46)] [timeMillis: 1717095474542] [levelValue: 900] [[
  StandardWrapperValve[default]: Servlet.service() for servlet default threw exception
java.io.IOException: Connection closed
	at org.glassfish.grizzly.asyncqueue.TaskQueue.onClose(TaskQueue.java:299)
	at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.onClose(AbstractNIOAsyncQueueWriter.java:415)
	at org.glassfish.grizzly.nio.transport.TCPNIOTransport.closeConnection(TCPNIOTransport.java:379)
	at org.glassfish.grizzly.nio.NIOConnection.doClose(NIOConnection.java:606)
	at org.glassfish.grizzly.nio.NIOConnection$6.run(NIOConnection.java:573)
	at org.glassfish.grizzly.nio.DefaultSelectorHandler.execute(DefaultSelectorHandler.java:190)
	at org.glassfish.grizzly.nio.NIOConnection.terminate0(NIOConnection.java:568)
	at org.glassfish.grizzly.nio.transport.TCPNIOConnection.terminate0(TCPNIOConnection.java:249)
	at org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:102)
	at org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:77)
	at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.processAsync(AbstractNIOAsyncQueueWriter.java:285)
	at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:92)
	at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:51)
	at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:510)
	at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:82)
	at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:83)
	at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.executeIoEvent(WorkerThreadIOStrategy.java:73)
	at org.glassfish.grizzly.strategies.AbstractIOStrategy.executeIoEvent(AbstractIOStrategy.java:66)
	at org.glassfish.grizzly.nio.SelectorRunner.iterateKeyEvents(SelectorRunner.java:381)
	at org.glassfish.grizzly.nio.SelectorRunner.iterateKeys(SelectorRunner.java:353)
	at org.glassfish.grizzly.nio.SelectorRunner.doSelect(SelectorRunner.java:319)
	at org.glassfish.grizzly.nio.SelectorRunner.run(SelectorRunner.java:248)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:535)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:515)
	at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.io.IOException: An established connection was aborted by the software in your host machine
	at java.base/sun.nio.ch.SocketDispatcher.write0(Native Method)
	at java.base/sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:54)
	at java.base/sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:137)
	at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:81)
	at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:58)
	at java.base/sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:542)
	at org.glassfish.grizzly.nio.transport.TCPNIOUtils.flushByteBuffer(TCPNIOUtils.java:118)
	at org.glassfish.grizzly.nio.transport.TCPNIOUtils.writeSimpleBuffer(TCPNIOUtils.java:104)
	at org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:94)
	... 16 more
]]

Thanks!

Marc

2

Hi,

do you use https?

Steven

3

Hi @StevenHachel - yes, we do use HTTPS.

4

Then it’s simple. Unfortunately, the Payara developers don’t really take care of this problem, which is really annoying.

Payara Administration Console:
You have to go to Configurations → server-config → Network Config → http-listener-2 (the configured https listener) → http → deactivate HTTP/2.

Then everything will work again and you won’t have constant crashes. With a very demanding JSF application, this can be disastrous if there are constant crashes. Containers are not reloaded, constant Javascript errors, incorrect page loading, etc…

Steven

5

Hi @StevenHachel - actually we disabled HTTP2 years ago, for reasons like you specify.

We aren’t actually having any application issues of any kind. These traces just show up periodically. My assumption is that these are just client disconnects from long file downloads or video playback, not actual errors. I was just hoping there was a clear way to suppress this particular stack trace as it makes it hard to find real errors in the server.log file.

6

Ah, okay. Too bad I couldn’t help.