UDP Ports leak on calling close on ApnsClient - 0.13.5

I have a code where I keep fixed size cache of APNS clients. On eviction from the cache, apns clients are closed.

final Future<Void> closeFuture = entry.getKey().getApnsClient().close().addListener(new GenericFutureListener<Future<Void>>() {
	@Override
	public void operationComplete(Future<Void> future) throws Exception {
		if (future.isSuccess()) {
		    logger.info("Pushy APNS closed call success");
                } else {
		   logger.info("Pushy APNS closed call failed");
                }
        } 
});

I have confirmed the number of clients evicted from cache are equal to number of clients on which close got called successfully. But noticed the number of udp ports opened by the java process keeps increasing very highly. On enabling pushy and netty logging it seems these ports are opened due to RoundRobinDnsAddressResolverGroup and on close these ports are not closing. This open port count only keeps increasing due to large volume of evictions and close and over a period of some days app becomes unresponsive and needs a restart.

lsof -n -p <pid> | sed -n '1p;/UDP/p' | wc -l

I made local changes to Pushy in ApnsChannelFactory:: this.bootstrapTemplate.resolver to use DefaultAddressResolverGroup.INSTANCE and number of open udp ports became stable.

So https://github.com/relayrides/pushy/issues/632 doesn’t seem to be fixed

[jchambers]

netty/netty#9214 has been merged and should go out as part of Netty 4.1.37.

[eugeneseppel]

@jchambers I solicit for reopening this ticket again. After several weeks running in production I’ve figured out that experience a memory leak. There is a unit tests that shows this leak: ApnsClientTest.java, add this one:

    @Test
    public void test1() throws Exception {
        final SimpleApnsPushNotification pushNotification
                = new SimpleApnsPushNotification("apns:1111111111111111111111111111111111111111111111111111111111111111",
                "com.twilio.www", "{}");

        NioEventLoopGroup group = new NioEventLoopGroup(2);
        for(int i = 0; ; i++) {
            final ApnsClient client = new ApnsClientBuilder()
                    .setApnsServer("www.something.com") // to not flood apns
                    .setSigningKey(signingKey)
                    .setEventLoopGroup(group)
                    .build();
            Thread.sleep(10);
            client.sendNotification(pushNotification).await();
            client.close().await();

            if (i % 50 == 0) {
                Runtime.getRuntime().gc();
            }

            System.out.printf("y=%d\n", i);
        }
    }

Please run this test until 16000 iterations with current version of pushy and run $ while true; do ps axu|grep test1|grep -v grep|awk ‘{print $5,$6}’; sleep 2; done in terminal to check RSS is increasing After that you can eliminate usage/creation of RoundRobinAddressResolver in ApnsChannelFactory:

remove private final AddressResolverGroup addressResolverGroup; this.bootstrapTemplate.resolver(proxyHandlerFactory == null ? DefaultAddressResolverGroup.INSTANCE : NoopAddressResolverGroup.INSTANCE);

rerun this test and see that RSS is quite stable.

Moreover I have to add that currently in production our project consumes ~100 open UDP ports for resolver. (previously was ~1). It is not a serious issue for our usage pattern but quite strange why one need 100 ports to resolve just 2 APNS domain names