Refactor to use the built-in LLM Logging and AI Foundry (#182)

* Refactor to use the built-in LLM Logging and AI Foundry

Author: vieiraae

labs/access-controlling/README.MD

@@ -1,20 +1,18 @@
-# APIM ❤️ OpenAI
+# APIM ❤️ AI Foundry
 
 ## [Access Controlling lab](access-controlling.ipynb)
 
 [![flow](../../images/access-controlling.gif)](access-controlling.ipynb)
 
-Playground to try the [OAuth 2.0 authorization feature](https://learn.microsoft.com/azure/api-management/api-management-authenticate-authorize-azure-openai#oauth-20-authorization-using-identity-provider) using identity provider to enable more fine-grained access to OpenAPI APIs by particular users or client.
+Playground to try the [OAuth 2.0 authorization feature](https://learn.microsoft.com/azure/api-management/api-management-authenticate-authorize-azure-openai#oauth-20-authorization-using-identity-provider) using identity provider to enable more fine-grained access to AI Foundry models by particular users or client.
 
 ### Prerequisites
 
 - [Python 3.12 or later version](https://www.python.org/) installed
-- [Pandas Library](https://pandas.pydata.org) installed
 - [VS Code](https://code.visualstudio.com/) installed with the [Jupyter notebook extension](https://marketplace.visualstudio.com/items?itemName=ms-toolsai.jupyter) enabled
-- [Azure CLI](https://learn.microsoft.com/cli/azure/install-azure-cli) installed
-- [An Azure Subscription](https://azure.microsoft.com/free/) with Contributor permissions
-- [Access granted to Azure OpenAI](https://aka.ms/oai/access) or just enable the mock service
-- [Sign in to Azure with Azure CLI](https://learn.microsoft.com/cli/azure/authenticate-azure-cli-interactively)
+- [Python environment](https://code.visualstudio.com/docs/python/environments#_creating-environments) with the [requirements.txt](../../requirements.txt) or run `pip install -r requirements.txt` in your terminal
+- [An Azure Subscription](https://azure.microsoft.com/free/) with [Contributor](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles/privileged#contributor) + [RBAC Administrator](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles/privileged#role-based-access-control-administrator) or [Owner](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles/privileged#owner) roles
+- [Azure CLI](https://learn.microsoft.com/cli/azure/install-azure-cli) installed and [Signed into your Azure subscription](https://learn.microsoft.com/cli/azure/authenticate-azure-cli-interactively)
 
 ### 🚀 Get started
 

labs/access-controlling/access-controlling.ipynb

@@ -4,21 +4,22 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# APIM ❤️ OpenAI\n",
+    "# APIM ❤️ AI Foundry\n",
     "\n",
     "## Access Controlling lab\n",
     "![flow](../../images/access-controlling.gif)\n",
     "\n",
-    "Playground to try the [OAuth 2.0 authorization feature](https://learn.microsoft.com/azure/api-management/api-management-authenticate-authorize-azure-openai#oauth-20-authorization-using-identity-provider) using identity provider to enable more fine-grained access to OpenAPI APIs by particular users or client.\n",
+    "Playground to try the [OAuth 2.0 authorization feature](https://learn.microsoft.com/azure/api-management/api-management-authenticate-authorize-azure-openai#oauth-20-authorization-using-identity-provider) using identity provider to enable more fine-grained access to AI Foundry models by particular users or client.\n",
     "\n",
     "### Prerequisites\n",
+    "\n",
     "- [Python 3.12 or later version](https://www.python.org/) installed\n",
-    "- [Pandas Library](https://pandas.pydata.org/) installed\n",
     "- [VS Code](https://code.visualstudio.com/) installed with the [Jupyter notebook extension](https://marketplace.visualstudio.com/items?itemName=ms-toolsai.jupyter) enabled\n",
-    "- [Azure CLI](https://learn.microsoft.com/cli/azure/install-azure-cli) installed\n",
-    "- [An Azure Subscription](https://azure.microsoft.com/free/) with Contributor permissions\n",
-    "- [Access granted to Azure OpenAI](https://aka.ms/oai/access) or just enable the mock service\n",
-    "- [Sign in to Azure with Azure CLI](https://learn.microsoft.com/cli/azure/authenticate-azure-cli-interactively)"
+    "- [Python environment](https://code.visualstudio.com/docs/python/environments#_creating-environments) with the [requirements.txt](../../requirements.txt) or run `pip install -r requirements.txt` in your terminal\n",
+    "- [An Azure Subscription](https://azure.microsoft.com/free/) with [Contributor](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles/privileged#contributor) + [RBAC Administrator](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles/privileged#role-based-access-control-administrator) or [Owner](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles/privileged#owner) roles\n",
+    "- [Azure CLI](https://learn.microsoft.com/cli/azure/install-azure-cli) installed and [Signed into your Azure subscription](https://learn.microsoft.com/cli/azure/authenticate-azure-cli-interactively)\n",
+    "\n",
+    "▶️ Click `Run All` to execute all steps sequentially, or execute them `Step by Step`... \n"
    ]
   },
   {
@@ -30,7 +31,7 @@
     "\n",
     "- Resources will be suffixed by a unique string based on your subscription id.\n",
     "- Adjust the location parameters according your preferences and on the [product availability by Azure region.](https://azure.microsoft.com/explore/global-infrastructure/products-by-region/?cdn=disable&products=cognitive-services,api-management) \n",
-    "- Adjust the OpenAI model and version according the [availability by region.](https://learn.microsoft.com/azure/ai-services/openai/concepts/models) "
+    "- Adjust the models and versions according the [availability by region.](https://learn.microsoft.com/azure/ai-services/openai/concepts/models) "
    ]
   },
   {
@@ -47,19 +48,21 @@
     "\n",
     "deployment_name = os.path.basename(os.path.dirname(globals()['__vsc_ipynb_file__']))\n",
     "resource_group_name = f\"lab-{deployment_name}\" # change the name to match your naming style\n",
-    "app_registration_name = f\"{deployment_name}-app\" # name of the app that will be registered in Microsoft Entra ID\n",
     "resource_group_location = \"uksouth\"\n",
     "\n",
+    "aiservices_config = [{\"name\": \"foundry1\", \"location\": \"uksouth\"}]\n",
+    "\n",
+    "models_config = [{\"name\": \"gpt-4o-mini\", \"publisher\": \"OpenAI\", \"version\": \"2024-07-18\", \"sku\": \"GlobalStandard\", \"capacity\": 100}]\n",
+    "\n",
     "apim_sku = 'Basicv2'\n",
+    "apim_subscriptions_config = [{\"name\": \"subscription1\", \"displayName\": \"Subscription 1\"}]\n",
     "\n",
-    "openai_resources = [ {\"name\": \"openai1\", \"location\": \"uksouth\"}]\n",
+    "inference_api_path = \"inference\"  # path to the inference API in the APIM service\n",
+    "inference_api_type = \"AzureOpenAI\"  # options: AzureOpenAI, AzureAI, OpenAI, PassThrough\n",
+    "inference_api_version = \"2025-03-01-preview\"\n",
+    "foundry_project_name = deployment_name\n",
     "\n",
-    "openai_model_name = \"gpt-4o-mini\"\n",
-    "openai_model_version = \"2024-07-18\"\n",
-    "openai_model_sku = \"GlobalStandard\"\n",
-    "openai_model_capacity = 20\n",
-    "openai_deployment_name = \"gpt-4o-mini\"\n",
-    "openai_api_version = \"2024-10-21\"\n",
+    "app_registration_name = f\"{deployment_name}-app\" # name of the app that will be registered in Microsoft Entra ID\n",
     "\n",
     "utils.print_ok('Notebook initialized')\n"
    ]
@@ -127,13 +130,12 @@
     "    \"contentVersion\": \"1.0.0.0\",\n",
     "    \"parameters\": {\n",
     "        \"apimSku\": { \"value\": apim_sku },\n",
-    "        \"openAIConfig\": { \"value\": openai_resources },\n",
-    "        \"openAIDeploymentName\": { \"value\": openai_deployment_name },\n",
-    "        \"openAIModelName\": { \"value\": openai_model_name },\n",
-    "        \"openAIModelVersion\": { \"value\": openai_model_version },\n",
-    "        \"openAIModelCapacity\": { \"value\": openai_model_capacity },\n",
-    "        \"openAIModelSKU\": { \"value\": openai_model_sku },\n",
-    "        \"openAIAPIVersion\": { \"value\": openai_api_version },\n",
+    "        \"aiServicesConfig\": { \"value\": aiservices_config },\n",
+    "        \"modelsConfig\": { \"value\": models_config },\n",
+    "        \"apimSubscriptionsConfig\": { \"value\": apim_subscriptions_config },\n",
+    "        \"inferenceAPIPath\": { \"value\": inference_api_path },\n",
+    "        \"inferenceAPIType\": { \"value\": inference_api_type },\n",
+    "        \"foundryProjectName\": { \"value\": foundry_project_name },\n",
     "        \"tenantId\": { \"value\": tenant_id },\n",
     "        \"clientId\": { \"value\": client_id }\n",
     "    }\n",
@@ -168,10 +170,16 @@
     "output = utils.run(f\"az deployment group show --name {deployment_name} -g {resource_group_name}\", f\"Retrieved deployment: {deployment_name}\", f\"Failed to retrieve deployment: {deployment_name}\")\n",
     "\n",
     "if output.success and output.json_data:\n",
+    "    log_analytics_id = utils.get_deployment_output(output, 'logAnalyticsWorkspaceId', 'Log Analytics Id')\n",
     "    apim_service_id = utils.get_deployment_output(output, 'apimServiceId', 'APIM Service Id')\n",
     "    apim_resource_gateway_url = utils.get_deployment_output(output, 'apimResourceGatewayURL', 'APIM API Gateway URL')\n",
-    "    apim_subscription_key = utils.get_deployment_output(output, 'apimSubscriptionKey', 'APIM Subscription Key (masked)', True)\n",
-    "\n"
+    "    apim_subscriptions = json.loads(utils.get_deployment_output(output, 'apimSubscriptions').replace(\"\\'\", \"\\\"\"))\n",
+    "    for subscription in apim_subscriptions:\n",
+    "        subscription_name = subscription['name']\n",
+    "        subscription_key = subscription['key']\n",
+    "        utils.print_info(f\"Subscription Name: {subscription_name}\")\n",
+    "        utils.print_info(f\"Subscription Key: ****{subscription_key[-4:]}\")\n",
+    "    api_key = apim_subscriptions[0].get(\"key\") # default api key to the first subscription key"
    ]
   },
   {
@@ -233,13 +241,19 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "import requests\n",
+    "import requests, base64, json\n",
     "\n",
     "result = app.acquire_token_by_device_flow(flow)\n",
     "\n",
     "if \"access_token\" in result:\n",
     "    access_token = result['access_token']\n",
     "    # Calling graph using the access token\n",
+    "    header, payload, signature = access_token.split('.')\n",
+    "    def pad(b): return b + '=' * (-len(b) % 4)\n",
+    "    print(\"Decoded JWT Header and Payload:\")\n",
+    "    print(json.dumps(json.loads(base64.urlsafe_b64decode(pad(header)).decode('utf-8')), indent=4))\n",
+    "    print(json.dumps(json.loads(base64.urlsafe_b64decode(pad(payload)).decode('utf-8')), indent=4))\n",
+    "\n",
     "    graph_data = requests.get(  # Use token to call downstream service\n",
     "        \"https://graph.microsoft.com/v1.0/me\",\n",
     "        headers={'Authorization': 'Bearer ' + access_token},).json()\n",
@@ -248,16 +262,16 @@
     "else:\n",
     "    print(result.get(\"error\"))\n",
     "    print(result.get(\"error_description\"))\n",
-    "    print(result.get(\"correlation_id\"))  # You may need this when reporting a bug"
+    "    print(result.get(\"correlation_id\"))  # You may need this when reporting a bug\n",
+    "\n"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
     "<a id='requests'></a>\n",
-    "### 🧪 Test the API using a direct HTTP call\n",
-    "Requests is an elegant and simple HTTP library for Python that will be used here to make raw API requests and inspect the responses.\n",
+    "### 🧪 Test the API using the access token\n",
     "\n",
     "Tip: Use the [tracing tool](../../tools/tracing.ipynb) to debug the policy."
    ]
@@ -268,14 +282,14 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "url = apim_resource_gateway_url + \"/openai/deployments/\" + openai_deployment_name + \"/chat/completions?api-version=\" + openai_api_version\n",
+    "url = f\"{apim_resource_gateway_url}/{inference_api_path}/openai/deployments/{models_config[0]['name']}/chat/completions?api-version={inference_api_version}\"\n",
     "\n",
     "messages = { \"messages\": [\n",
     "    {\"role\": \"system\", \"content\": \"You are a sarcastic unhelpful assistant.\"},\n",
     "    {\"role\": \"user\", \"content\": \"Can you tell me the time, please?\"}\n",
     "]}\n",
     "\n",
-    "response = requests.post(url, headers = {'api-key': apim_subscription_key, 'Authorization': 'Bearer ' + access_token}, json = messages)\n",
+    "response = requests.post(url, headers = {'api-key': api_key, 'Authorization': 'Bearer ' + access_token}, json = messages)\n",
     "utils.print_response_code(response)\n",
     "\n",
     "if (response.status_code == 200):\n",
@@ -304,12 +318,12 @@
     "from openai import AzureOpenAI\n",
     "\n",
     "client = AzureOpenAI(\n",
-    "    azure_endpoint = apim_resource_gateway_url,\n",
-    "    api_key = apim_subscription_key,\n",
-    "    api_version = openai_api_version\n",
+    "    azure_endpoint = f\"{apim_resource_gateway_url}/{inference_api_path}\",\n",
+    "    api_key = api_key,\n",
+    "    api_version = inference_api_version\n",
     ")\n",
     "\n",
-    "response = client.chat.completions.create(model = openai_model_name, messages = [\n",
+    "response = client.chat.completions.create(model = models_config[0]['name'], messages = [\n",
     "                        {\"role\": \"system\", \"content\": \"You are a sarcastic unhelpful assistant.\"},\n",
     "                        {\"role\": \"user\", \"content\": \"Can you tell me the time, please?\"}\n",
     "                    ], \n",
@@ -346,7 +360,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.12.9"
+   "version": "3.12.10"
   }
  },
  "nbformat": 4,

labs/access-controlling/clean-up-resources.ipynb

@@ -65,7 +65,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.12.9"
+   "version": "3.12.10"
   }
  },
  "nbformat": 4,

labs/access-controlling/main.bicep

@@ -2,66 +2,79 @@
 //    PARAMETERS
 // ------------------
 
-// Typically, parameters would be decorated with appropriate metadata and attributes, but as they are very repetetive in these labs we omit them for brevity.
-
+param aiServicesConfig array = []
+param modelsConfig array = []
 param apimSku string
-param openAIConfig array = []
-param openAIModelName string
-param openAIModelVersion string
-param openAIDeploymentName string
-param openAIModelSKU string
-param openAIModelCapacity int
-param openAIAPIVersion string
+param apimSubscriptionsConfig array = []
+param inferenceAPIType string = 'AzureOpenAI'
+param inferenceAPIPath string = 'inference' // Path to the inference API in the APIM service
+param foundryProjectName string = 'default'
 param tenantId string
 param clientId string
 
 // ------------------
-//    VARIABLES
+//    RESOURCES
 // ------------------
 
-// Account for all placeholders in the polixy.xml file.
-var policyXml = loadTextContent('policy.xml')
-var updatedPolicyXml = replace(replace(replace(policyXml, '{backend-id}', (length(openAIConfig) > 1) ? 'openai-backend-pool' : openAIConfig[0].name), '{tenant-id}', tenantId), '{client-application-id}', clientId)
+// 1. Log Analytics Workspace
+module lawModule '../../modules/operational-insights/v1/workspaces.bicep' = {
+  name: 'lawModule'
+}
 
-// ------------------
-//    RESOURCES
-// ------------------
+// 2. Application Insights
+module appInsightsModule '../../modules/monitor/v1/appinsights.bicep' = {
+  name: 'appInsightsModule'
+  params: {
+    lawId: lawModule.outputs.id
+    customMetricsOptedInType: 'WithDimensions'
+  }
+}
 
-// 1. API Management
-module apimModule '../../modules/apim/v1/apim.bicep' = {
+// 3. API Management
+module apimModule '../../modules/apim/v2/apim.bicep' = {
   name: 'apimModule'
   params: {
     apimSku: apimSku
+    apimSubscriptionsConfig: apimSubscriptionsConfig
+    lawId: lawModule.outputs.id
+    appInsightsId: appInsightsModule.outputs.id
+    appInsightsInstrumentationKey: appInsightsModule.outputs.instrumentationKey
   }
 }
 
-// 2. Cognitive Services
-module openAIModule '../../modules/cognitive-services/v1/openai.bicep' = {
-  name: 'openAIModule'
-  params: {
-    openAIConfig: openAIConfig
-    openAIDeploymentName: openAIDeploymentName
-    openAIModelName: openAIModelName
-    openAIModelVersion: openAIModelVersion
-    openAIModelSKU: openAIModelSKU
-    openAIModelCapacity: openAIModelCapacity
-    apimPrincipalId: apimModule.outputs.principalId
+// 4. AI Foundry
+module foundryModule '../../modules/cognitive-services/v3/foundry.bicep' = {
+    name: 'foundryModule'
+    params: {
+      aiServicesConfig: aiServicesConfig
+      modelsConfig: modelsConfig
+      apimPrincipalId: apimModule.outputs.principalId
+      foundryProjectName: foundryProjectName
+    }
   }
-}
 
-// 3. APIM OpenAI API
-module openAIAPIModule '../../modules/apim/v1/openai-api.bicep' = {
-  name: 'openAIAPIModule'
+// 5. APIM Inference API
+module inferenceAPIModule '../../modules/apim/v2/inference-api.bicep' = {
+  name: 'inferenceAPIModule'
   params: {
-    policyXml: updatedPolicyXml
-    openAIConfig: openAIModule.outputs.extendedOpenAIConfig
-    openAIAPIVersion: openAIAPIVersion
+    policyXml: replace(replace(loadTextContent('policy.xml'), '{tenant-id}', tenantId), '{client-application-id}', clientId)
+    apimLoggerId: apimModule.outputs.loggerId
+    aiServicesConfig: foundryModule.outputs.extendedAIServicesConfig
+    inferenceAPIType: inferenceAPIType
+    inferenceAPIPath: inferenceAPIPath
   }
 }
 
+
 // ------------------
-//    MARK: OUTPUTS
+//    OUTPUTS
 // ------------------
+
+output logAnalyticsWorkspaceId string = lawModule.outputs.customerId
 output apimServiceId string = apimModule.outputs.id
 output apimResourceGatewayURL string = apimModule.outputs.gatewayUrl
-output apimSubscriptionKey string = openAIAPIModule.outputs.subscriptionPrimaryKey
+
+output apimSubscriptions array = apimModule.outputs.apimSubscriptions
+
+
+