[BUG] Hardcoded Endpoint for OpenAI

Library name and version

Azure.AI.OpenAI_1.0.0-beta.6

Describe the bug

Hi Team!

Companies create internal API proxies to avoid direct calls to https://api.openai.com/ . But Azure has OpenAI endpoint hardcoded https://github.com/Azure/azure-sdk-for-net/blob/c8b27f9fca17384a481a1debf565052ffc97b2b9/sdk/openai/Azure.AI.OpenAI/src/Custom/OpenAIClient.cs#L28 without possibility to specify custom endpoint. Constructors use only hardcoded PublicOpenAIEndpoint property https://github.com/Azure/azure-sdk-for-net/blob/c8b27f9fca17384a481a1debf565052ffc97b2b9/sdk/openai/Azure.AI.OpenAI/src/Custom/OpenAIClient.cs#L120-L131

Expected behavior

It should be possible to specify a custom endpoint. Maybe add one more constructor with _nonAzureOpenAIApiKey = openAIApiKey; code. See https://github.com/Azure/azure-sdk-for-net/blob/c8b27f9fca17384a481a1debf565052ffc97b2b9/sdk/openai/Azure.AI.OpenAI/src/Custom/OpenAIClient.cs#L120-L131

Actual behavior

Constructors use only hardcoded PublicOpenAIEndpoint property https://github.com/Azure/azure-sdk-for-net/blob/c8b27f9fca17384a481a1debf565052ffc97b2b9/sdk/openai/Azure.AI.OpenAI/src/Custom/OpenAIClient.cs#L120-L131

Reproduction Steps

OpenAIClient client = new OpenAIClient("your-api-key-from-platform.openai.com");

Response<Completions> response = await client.GetCompletionsAsync(
    "text-davinci-003", // assumes a matching model deployment or model name
    "Hello, world!");

foreach (Choice choice in response.Value.Choices)
{
    Console.WriteLine(choice.Text);
}

Environment

No response

[trrwilson]

@gingters – fully understood about environment-variable-based configuration not being viable.

From that second link I shared, there’s also a programmatic mechanism to configure a proxy server via client options; pulling that page together with the OpenAI code, it’d be along the lines of:

var httpClientHandler = new HttpClientHandler()
{
    Proxy = new WebProxy("http://my-enterprise.proxy-server.com"),
};
var httpClientWithProxy = new HttpClient(httpClientHandler);
var openAIClientOptions = new OpenAIClientOptions()
{
    Transport = new HttpClientTransport(httpClientWithProxy),
};
var client = new OpenAIClient(
    endpoint: new Uri("https://destination-aoai-resource.openai.azure.com"),
    new AzureKeyCredential("fake-key-to-be-replaced-by-proxy"),
    openAIClientOptions);

This will configure the OpenAIClient instance to use the underlying HttpClient provided, which has in turn been configured (in code, sans environment variable involvement) to use a proxy.

I’ve scheduled some work for us to add better documentation and test automation coverage to clarify this, but in the interim I’ve validated by hand that this does indeed invoke a proxy request.

[victor-v-rad]

@trrwilson Thank you for sharing the proxy configuration. It works for me.