[QUERY] Azure SDK JAVA - How to use ProxyOptions to connect to the proxy URL and how to pass the required path to the httpClient along with the hostname

Query/Question Hi,

I am trying utilise the ProxyOptions to re-route the blob upload to a proxy URL. In this case localhost

Demo Project:

Gradle Dependency:

implementation 'com.azure:azure-storage-blob' 
implementation 'com.azure:azure-core' 
implementation 'com.azure:azure-core-http-netty' 
implementation 'com.azure:azure-identity' 
implementation 'com.azure:azure-security-keyvault-secrets' 

implementation ('com.azure.resourcemanager:azure-resourcemanager:2.18.0') { 
    exclude group: 'com.azure', module: 'azure-core' 
} 

configurations.all{ 
    resolutionStrategy{ 
        force 'com.azure:azure-storage-common:12.18.0', 'com.azure:azure-storage-common:12.14.3' 
    } 
} 

BlobClient Bean:

import com.azure.core.http.HttpClient;
import com.azure.core.http.ProxyOptions;
import com.azure.core.http.netty.NettyAsyncHttpClientBuilder;
import com.azure.storage.blob.BlobClient;
import com.azure.storage.blob.BlobClientBuilder;
import com.azure.storage.common.StorageSharedKeyCredential;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import java.net.InetSocketAddress;
import java.net.URISyntaxException;
import java.net.UnknownHostException;

@Bean 
    public BlobClient blobClient() throws URISyntaxException, UnknownHostException { 
        HttpClient client = new NettyAsyncHttpClientBuilder() 
                .proxy(new ProxyOptions(ProxyOptions.Type.HTTP, new InetSocketAddress("localhost", 8080))) 
                .build(); 
        String accountName = "xxxx"; 
        String accountkey = "yyyyy"; 
        String endPoint = String.format(java.util.Locale.ROOT, "https://%s.blob.core.windows.net", accountName); 
        BlobClient blobClient = new BlobClientBuilder() 
                .endpoint(endPoint) 
                .credential(new StorageSharedKeyCredential(accountName, accountkey)) 
                .containerName("test-bucket-1") 
                .blobName("test.txt") 
                .httpClient(client) 
                .buildClient(); 
        return blobClient; 
    } 

and the controller which consumes the above blobClient is :

import com.azure.core.http.rest.Response;
import com.azure.core.util.BinaryData;
import com.azure.core.util.Context;
import com.azure.storage.blob.BlobClient;
import com.azure.storage.blob.models.BlockBlobItem;
import com.azure.storage.blob.options.BlobParallelUploadOptions;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PutMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.io.InputStream;
import java.net.URISyntaxException;
import java.time.Duration;
import java.util.Optional;

@PutMapping(value ={"/azure/test"}) 
    public boolean uploadAzureBlob( 
            @RequestParam("containerName") String containerName, 
            @RequestParam("blobName") Optional<String> blobName, 
            HttpServletRequest request, HttpServletResponse response) throws IOException, URISyntaxException { 
        InputStream inputStream = request.getInputStream(); 
        BinaryData binaryData = BinaryData.fromStream(inputStream); 
        BlobParallelUploadOptions blobParallelUploadOptions = new BlobParallelUploadOptions(binaryData); 
        Response<BlockBlobItem> blobItemResponse = 
                blobClient.uploadWithResponse( 
                        blobParallelUploadOptions, 
                        Duration.ofSeconds(120L), 
                        Context.NONE); 
        return true; 
} 

What i am expecting is :

1. When the blobClient is trying to upload the blob object it should re-route to the following controller which is running in my local machine. I want to basically intercept the API call and redirect it to the proxy application running in my local. At least to the @RequestMapping(value ={“/*”})

Proxy Project:

Gradle Dependency:

implementation platform('com.azure:azure-sdk-bom:1.1.1') 
implementation 'com.azure:azure-storage-blob' 
implementation 'com.azure:azure-core' 
implementation 'com.azure:azure-core-http-netty' 
implementation 'com.azure:azure-identity' 
implementation 'com.azure:azure-security-keyvault-secrets' 

implementation ('com.azure.resourcemanager:azure-resourcemanager:2.18.0') { 
    exclude group: 'com.azure', module: 'azure-core' 
} 

configurations.all{ 
    resolutionStrategy{ 
        force 'com.azure:azure-storage-common:12.18.0', 'com.azure:azure-storage-common:12.14.3' 
    } 
} 

Controller Code:

@RequestMapping("/v2/azure-proxy") 
public class AzureBlobProxyController { 
    @PutMapping(value ={"/{containerName}/{blobName}"}) 
    public boolean uploadSecureStream(@PathVariable("containerName") String containerName, 
                                               @PathVariable("blobName") String blobName, 
                                               HttpServletRequest request, HttpServletResponse response) throws Exception { 
                return true; 
        } 

    @RequestMapping(value ={"/*"}) 
    public boolean getBlobContainerAsyncClient( 
                                            HttpServletRequest request, 
                                            HttpServletResponse httpServletResponse) throws Exception { 

       return true; 
    } 
} 

But i am getting the following exception:

2022-11-28 00:22:12.535 WARN 86304 — [r-http-kqueue-1] r.netty.http.client.HttpClientConnect : [c80a00e1, L:/127.0.0.1:54822 - R:localhost/127.0.0.1:8080] The connection observed an error

javax.net.ssl.SSLException: failure when writing TLS control frames at io.netty.handler.ssl.SslHandler.setHandshakeFailureTransportFailure(SslHandler.java:1896) ~[netty-handler-4.1.77.Final.jar:4.1.77.Final] at io.netty.handler.ssl.SslHandler.access$600(SslHandler.java:168) ~[netty-handler-4.1.77.Final.jar:4.1.77.Final] at io.netty.handler.ssl.SslHandler$2.operationComplete(SslHandler.java:933) ~[netty-handler-4.1.77.Final.jar:4.1.77.Final] at io.netty.handler.ssl.SslHandler$2.operationComplete(SslHandler.java:928) ~[netty-handler-4.1.77.Final.jar:4.1.77.Final] at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578) ~[netty-common-4.1.77.Final.jar:4.1.77.Final] at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:552) ~[netty-common-4.1.77.Final.jar:4.1.77.Final] at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491) ~[netty-common-4.1.77.Final.jar:4.1.77.Final] at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616) ~[netty-common-4.1.77.Final.jar:4.1.77.Final] at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609) ~[netty-common-4.1.77.Final.jar:4.1.77.Final] at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117) ~[netty-common-4.1.77.Final.jar:4.1.77.Final] at io.netty.channel.PendingWriteQueue.safeFail(PendingWriteQueue.java:288) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final] at io.netty.channel.PendingWriteQueue.removeAndFailAll(PendingWriteQueue.java:186) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final] at io.netty.handler.proxy.ProxyHandler.failPendingWrites(ProxyHandler.java:434) ~[netty-handler-proxy-4.1.77.Final.jar:4.1.77.Final] at io.netty.handler.proxy.ProxyHandler.failPendingWritesAndClose(ProxyHandler.java:351) ~[netty-handler-proxy-4.1.77.Final.jar:4.1.77.Final] at io.netty.handler.proxy.ProxyHandler.setConnectFailure(ProxyHandler.java:346) ~[netty-handler-proxy-4.1.77.Final.jar:4.1.77.Final] at io.netty.handler.proxy.ProxyHandler.channelRead(ProxyHandler.java:266) ~[netty-handler-proxy-4.1.77.Final.jar:4.1.77.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final] at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final] at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:327) ~[netty-codec-4.1.77.Final.jar:4.1.77.Final] at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:299) ~[netty-codec-4.1.77.Final.jar:4.1.77.Final] at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final] at io.netty.handler.proxy.HttpProxyHandler$HttpClientCodecWrapper.channelRead(HttpProxyHandler.java:272) ~[netty-handler-proxy-4.1.77.Final.jar:4.1.77.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final] at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final] at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final] at io.netty.channel.kqueue.AbstractKQueueStreamChannel$KQueueStreamUnsafe.readReady(AbstractKQueueStreamChannel.java:544) ~[netty-transport-classes-kqueue-4.1.77.Final.jar:4.1.77.Final] at io.netty.channel.kqueue.AbstractKQueueChannel$AbstractKQueueUnsafe.readReady(AbstractKQueueChannel.java:383) ~[netty-transport-classes-kqueue-4.1.77.Final.jar:4.1.77.Final] at io.netty.channel.kqueue.KQueueEventLoop.processReady(KQueueEventLoop.java:211) ~[netty-transport-classes-kqueue-4.1.77.Final.jar:4.1.77.Final] at io.netty.channel.kqueue.KQueueEventLoop.run(KQueueEventLoop.java:289) ~[netty-transport-classes-kqueue-4.1.77.Final.jar:4.1.77.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:995) ~[netty-common-4.1.77.Final.jar:4.1.77.Final] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.77.Final.jar:4.1.77.Final] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.77.Final.jar:4.1.77.Final] at java.base/java.lang.Thread.run(Thread.java:832) ~[na:na] Caused by: io.netty.handler.proxy.HttpProxyHandler$HttpProxyConnectException: http, none, localhost/127.0.0.1:8080 => xxxx.blob.core.windows.net/<unresolved>:443, status: 400
at io.netty.handler.proxy.HttpProxyHandler.handleResponse(HttpProxyHandler.java:200) ~[netty-handler-proxy-4.1.77.Final.jar:4.1.77.Final] at io.netty.handler.proxy.ProxyHandler.channelRead(ProxyHandler.java:257) ~[netty-handler-proxy-4.1.77.Final.jar:4.1.77.Final] … 23 common frames omitted

Caused by: io.netty.handler.proxy.HttpProxyHandler$HttpProxyConnectException: http, none, localhost/127.0.0.1:8080 => xxxx.blob.core.windows.net/<unresolved>:443, status: 400

• Kindly let me know how to fix the above issue and how can i make the proxy reach the @RequestMapping(“/v2/azure-proxy”)
• In the socketAddress i am able to give hostname how can i give the path variables?

Why is this not a Bug or a feature Request? It should have worked with the version of the SDK since i can see various posts stating that it works.

Setup (please complete the following information if applicable):

• OS: [iOS]
• IDE: [intelliJ]
• implementation platform(‘com.azure:azure-sdk-bom:1.1.1’) implementation ‘com.azure:azure-storage-blob’ implementation ‘com.azure:azure-core’ implementation ‘com.azure:azure-core-http-netty’ implementation ‘com.azure:azure-identity’ implementation ‘com.azure:azure-security-keyvault-secrets’

implementation (‘com.azure.resourcemanager:azure-resourcemanager:2.18.0’) { exclude group: ‘com.azure’, module: ‘azure-core’ }

configurations.all{ resolutionStrategy{ force ‘com.azure:azure-storage-common:12.18.0’, ‘com.azure:azure-storage-common:12.14.3’ } }

Information Checklist Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

• Query Added

• Setup information Added

@alzimmermsft

[syedyasiraswath]

Yes @alzimmermsft when I meant by multi-part is i meant the Content-Type: multipart/form-data with the part named as ‘file’

[alzimmermsft]

In this case I would create a custom HttpPipelinePolicy

public class RewriteRequestUrlToAzureProxy implements HttpPipelinePolicy {
    @Override
    public Mono<HttpResponse> process(HttpPipelineCallContext context, HttpPipelineNextPolicy next) {
        URL originalUrl = context.getHttpRequest().getUrl();
        // Do what you need to do to the URL.

        context.getHttpRequest().setUrl(updatedUrl);
        return next.process();
    }
}

BlobClient blobClient = new BlobClientBuilder()
                .endpoint(storageEndpoint)
                .containerName("azure-proxy-test-bucket-1")
                .blobName("test1.txt")
                .addPolicy(new RewriteRequestUrlToAzureProxy())
                .buildClient();
        return blobClient;

There will need to be more logic added from your side, but the premise is that this will rewrite the request URL to send it to your controller in a way it can consume and properly send the request on to Azure Storage. If needed you can set a custom HTTP header to the original URL to make this easier, such as

context.getHttpRequest().setHeader("a-custom-header-name-holding-original-request-url", originalUrl);

and the controller can use this header to reset the original URL after receiving the request to forward.