ClassCastException in Feign's default Client implementation, when UrlStreamHandlerFactory is overwritten

While running a job via spark/hadoop, before doing that, i fetch the job arguments from a remote service using Feign 9.5.0 (to get out of the way of the jar dependency hell within spark/hadoop)

When the client connect, i get this:

7/12/19 20:07:28 ERROR MachineLearningJob$: Error during block list computation: java.lang.ClassCastException: org.apache.hadoop.fs.FsUrlConnection cannot be cast to java.net.HttpURLConnection at feign.Client$Default.convertAndSend(Client.java:80) at feign.Client$Default.execute(Client.java:73) at feign.SynchronousMethodHandler.executeAndDecode(SynchronousMethodHandler.java:97) at feign.SynchronousMethodHandler.invoke(SynchronousMethodHandler.java:76) at feign.ReflectiveFeign$FeignInvocationHandler.invoke(ReflectiveFeign.java:103) ...

The Feign-Client code looks like this:

HttpURLConnection convertAndSend(Request request, Options options) throws IOException { final HttpURLConnection connection = (HttpURLConnection) new URL(request.url()).openConnection(); ...

So it looks like, spark/hadoop does set its own urlstreamhandlerfactory for http/https, that opens its own URLConnection implementation, which are not of Http(s)UrlConnection (https://issues.apache.org/jira/browse/HADOOP-14598) But this is what Feign’s default client assumes.

My suggestion is to extend checking your assumptions and if they are not met, to get out of the way with a proper exception. Maybe a hint, that someone has set its own handler. Maybe with a hint, that there are other client implementation (feign-okhttp, feign-httpclient), just something to speed up debugging, when this occurs. Maybe documenting it in the source code is also enough. This might save someone else’s lifetime!

What do you think?

Happy holidays

[okrische]

It is a bug. Feign’s default client assumes, that URL#openConnection returns HttpUrlConnection. Which is not true, if someone (in my case it was spark/hadoop) has overwritten the default factory within URL.

[okrische]

(It is still a bug, though, and it can appear at any time again. Suggestion was to go out of the way with a proper exception, if expectation does not meet reality: Feign Client only works, if factory returns Http(s)UrlConnection.)