Netty IdleStateHandler times out for large messages

Netty IdleStateHandler times out for large messages

I'm using Netty's IdleStateHandler in my client/server communication to
send heartbeats (server to client) and detect timeouts on the client. This
works fine expect for rare cases when the transfer of large messages takes
longer than the configured read timeout. I my case these large messages
only happen at the client's startup (baseline), during normal operation
the messages are small, hence I'm reluctant to increase the overall read
timeout on the client.
Is there a way for prevent the IdleStateHandler from firing idle state
events while it's actually receiving data? Or am I doing something wrong?
Thanks, Thomas