Azure Open AI in VNet

GPT models are hosted in multiple service vendor at the moment, and Microsoft Azure is one of them.

Even though the models themselves are the same, there are many differences including:

• cost
• functionalities
• type of models and versions
• geo location
• security
• support
• etc.

One of the most important aspects when we use it in an Enterprise Environment is, of course, security.

By using Azure network security features with Azure Open AI, customers can consume the Open AI service from and within the VNet, therefore no information is flowing in public.

Sample Deployment

Azure Sample repo provides a sample bicep files to deploy Azure Open AI into VNet environment.

GitHub: openai-enterprise-iac

The key features the bicep uses are:

• VNet
• VNet integration for Web App
• Private Endpoint for Azure Open AI
• Private Endpoint for Cognitive Search
• Private DNS Zone

By using these features, all the outbound traffic from the Web App only routed inside the VNet and all the names are resolved into private IP addresses. Open AI and Cognitive Search shut down the public IP address, thus there is not public interface endpoint available anymore.

Deploy

The bicep file will deploy following Azure Resources.

Let's deploy and confirm how it works. I create a resource group in East US region for my own test.

git clone https://github.com/Azure-Samples/openai-enterprise-iac
cd openai-enterprise-iac
az group create -n openaitest -l eastus
az deployment group create -g openaitest -f .\infra\main.bicep

Once I run the commend above, I see the deployment started.

Wait until the deployment completes.

Test

Let's see if the deployment was succeeded.

Azure Open AI

Let's try public access first.

I could create a deployment without any issue. But when I try from the Chat playground in my Azure Portal, I see the following error.

How about access via the Web API?

From an advanced tool of the App Service, I login to Bash session, and first I ping the service URL.

I see the private IP address assigned to the Private Endpoint is returend.

Then I use curl command to send request to the endpoint.