DEV Community

Cover image for Azure Synapse Analytics(workspaces): Deploy and Debug - Part 2
Jayendran Arumugam
Jayendran Arumugam

Posted on

Azure Synapse Analytics(workspaces): Deploy and Debug - Part 2

#azure #azuresynapseanalytics #cloud #api

Introduction:

In the previous post, we saw the basics architecture and understanding of the ARM template and parameters of synapse analytics workspace. In this post, we are going to see how to secure your synapse analytics workspace by giving proper permission through APIs.

Different types of APIs:

Managing Azure Synapse workspace can be possible with two different REST APIs

  • Management API
  • Data Plane API

Usually for all the azure resources we commonly use a REST API which known as Management API. In the case of synapse workspace, we have an additional one special API called Data Plane API.

Alt Text

Management API:

  • The REST APIs to create and manage Azure Synapses resources through Azure Resource Manager(ARM)

  • Mainly used for management operations such as create,update,delete synapse workspace.

  • The {api-version} should be 2019-06-01-preview

  • The audience claim (used for obtaining bearer token -Authorization) should be "https://management.core.windows.net" or "https://management.azure.com"

  • The Base API Endpoint looks like

https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Synapse/workspaces/{workspaceName}
Enter fullscreen mode Exit fullscreen mode

Data Plane API:

  • The REST APIs to create and manage Azure Synapses resources through individual Azure synapse workspace endpoint itself.

  • Used for managing individual synapse workspace operations such as workspace role-assignments,managing and monitoring spark and sql jobs,dataflows,pipelines,datasets,linkedservices,triggers and notebooks.

  • The {api-version} should be 2019-11-01-preview or 2020-02-01-preview

  • The audience claim (used for obtaining bearer token -Authorization) should be "https://dev.azuresynapse.net"

  • The Base API Endpoint looks like

https://<workspacename>.dev.azuresynapse.net/
Enter fullscreen mode Exit fullscreen mode

Right now there is no docs available for Data Plane API(preview). However you can get information from Github docs

Listing available synapse workspace - Management API

Lets see how we can call the management API to list a synapse workspace in a resource group.

Note: You can also directly try these API in the azure docs. Using Try It Option like below

Alt Text

However the above method use user impersonation and not client credentials (using SPN) to grab the bearer token. So lets see how we can call this API using client credentials method.

Getting Bearer token

We are going to use client_credentials way of using SPN(client_id/client_secret) to get the JWT Token.
The below is the simple curl command to invoke the Authorization API and obtain the bearer token.

Note: Make sure that your SPN have proper RBAC role for your purpose. In this below example my SPN have Contributor access to the Resource Group

curl --request POST \
  --url https://login.microsoftonline.com/<tenant-id>/oauth2/token \
  --header 'accept:  application/json' \
  --header 'content-type: multipart/form-data;' \
  --form client_id=a35373d8-c772-4ea0-9f4b-73111376354f \
  --form 'client_secret=xxxxxxx~~2Z6Es' \
  --form grant_type=client_credentials \
  --form resource=https://management.azure.com/ \
  --form scope=Microsoft.Synapse/workspaces/read
Enter fullscreen mode Exit fullscreen mode

Response:

{
  "token_type": "Bearer",
  "expires_in": "3599",
  "ext_expires_in": "3599",
  "expires_on": "1599917407",
  "not_before": "1599913507",
  "resource": "https://management.azure.com/",
  "access_token": "eyJ....."
}
Enter fullscreen mode Exit fullscreen mode

Insomnia Screenshot:

Alt Text

Calling Synapse management workspace List API

Now we got the token, lets call the actual API

curl --request GET \
  --url 'https://management.azure.com/subscriptions/<sub-id>/resourceGroups/azuresynapses/providers/Microsoft.Synapse/workspaces/?api-version=2019-06-01-preview' \
  --header 'accept:  application/json' \
  --header 'authorization: Bearer eyJ0......................' \
  --header 'content-type: application/json'
Enter fullscreen mode Exit fullscreen mode

Response:

{
  "value": [
    {
      "id": "/subscriptions/<sub-id>/resourceGroups/azuresynapses/providers/Microsoft.Synapse/workspaces/azsynapse002",
      "location": "eastus",
      "name": "azsynapse002",
      "type": "Microsoft.Synapse/workspaces",
      "identity": {
        "type": "SystemAssigned",
        "principalId": "712cc76e-7dd9-4978-a259-6c2be5057d2f",
        "tenantId": "<tenant-id>"
      },
      "tags": {},
      "properties": {
        "connectivityEndpoints": {
          "web": "https://web.azuresynapse.net?workspace=%2fsubscriptions%2f<sub-id>%2fresourceGroups%2fazuresynapses%2fproviders%2fMicrosoft.Synapse%2fworkspaces%2fazsynapse002",
          "sql": "azsynapse002.sql.azuresynapse.net",
          "dev": "https://azsynapse002.dev.azuresynapse.net",
          "sqlOnDemand": "azsynapse002-ondemand.sql.azuresynapse.net"
        },
        "managedResourceGroupName": "azuresynapses",
        "privateEndpointConnections": [],
        "workspaceUID": "0fdd8032-3277-4d1b-b4c3-b069f48bd169",
        "extraProperties": {
          "IsScopeEnabled": false
        },
        "provisioningState": "Succeeded"
      }
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

Insomnia Screenshot:

Alt Text

Synapse Workspace Roles

Before going to the Data Plane API. Let see what is the synapse workspace roles 🤔. Because, we are going to use Data Plane API to manage the workspace roles.

There are actually 3 different roles that are unique to Synapse and aren't based on Azure roles, which are

  • Synapse workspace admin
  • Synapse SQL admin
  • Apache Spark for Azure Synapse Analytics admin

There is an existing azure docs which has explained this in detail about the 3 different roles.

Alt Text

Managing workspace Role access - Data Plane API:

Now we understood the different roles in the synapse workspace. Lets see how we can manage these role access through Data Plane API.

Getting Bearer token

As usual we are going to use the SPN Authentication for getting the bearer token. Here a couple of differences are the

  • The resource param will be https://dev.azuresynapse.net
  • We no longer needed the scope param

Note: Make sure that your SPN is already a part of Workspace Admin.

curl --request POST \
  --url https://login.microsoftonline.com/<tenant-id>/oauth2/token \
  --header 'accept:  application/json' \
  --header 'content-type: multipart/form-data;' \
  --form client_id=a35373d8-c772-4ea0-9f4b-73111376354f \
  --form 'client_secret=xxxxxxx~~2Z6Es' \
  --form grant_type=client_credentials \
  --form resource=https://dev.azuresynapse.net
Enter fullscreen mode Exit fullscreen mode

Response:

{
  "token_type": "Bearer",
  "expires_in": "3599",
  "ext_expires_in": "3599",
  "expires_on": "1599917407",
  "not_before": "1599913507",
  "resource": "https://dev.azuresynapse.net",
  "access_token": "eyJ....."
}
Enter fullscreen mode Exit fullscreen mode

Insomnia Screenshot:

Alt Text

Calling Synapse Data Plane API to Add users to workspace roles

Here we are going to add a user into one of the 3 roles. In order to do so, we need to perform 2 operations

  1. Get the role id
  2. Add the user (object id) to that role id.

Get the role Id

In order to get the role id we have to use the below endpoint

https://<workspacename>.dev.azuresynapse.net/rbac/roles?api-version=2020-02-01-preview
Enter fullscreen mode Exit fullscreen mode

Lets curl it

curl --request GET \
  --url 'https://azsynapse002.dev.azuresynapse.net/rbac/roles?api-version=2020-02-01-preview' \
  --header 'accept: application/json' \
  --header 'authorization: Bearer eyJ0' \
  --header 'content-type: application/json'
Enter fullscreen mode Exit fullscreen mode

Response:

{
  "value": [
    {
      "id": "6e4bf58a-b8e1-4cc3-bbf9-d73143322b78",
      "name": "Workspace Admin",
      "isBuiltIn": true
    },
    {
      "id": "c3a6d2f1-a26f-4810-9b0f-591308d5cbf1",
      "name": "Apache Spark Admin",
      "isBuiltIn": true
    },
    {
      "id": "7af0c69a-a548-47d6-aea3-d00e69bd83aa",
      "name": "Sql Admin",
      "isBuiltIn": true
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

Insomnia Screenshot:
Alt Text

Add Users to the Role ID

Now we got the role id for each roles (these role ids are same for all the synapse workspace globally). Let add the user using below endpoint

https://<workspacename>.dev.azuresynapse.net/rbac/roleAssignments?api-version=2020-02-01-preview

#Json body Param:

{
"roleId": "<workspace role id>",
"principalId": "<objectid of the user/group>"
}
Enter fullscreen mode Exit fullscreen mode

Curl:

curl --request POST \
  --url 'https://azsynapse002.dev.azuresynapse.net/rbac/roleAssignments?api-version=2020-02-01-preview' \
  --header 'authorization: Bearer eyJ..............' \
  --header 'content-type: application/json' \
  --data '{
    "roleId": "6e4bf58a-b8e1-4cc3-bbf9-d73143322b78",
 "principalId": "fb1e7804-9542-4412-be66-e143a10e3b1a"
}'
Enter fullscreen mode Exit fullscreen mode

Response:

{
  "id": "6e4bf58a-b8e1-4cc3-bbf9-d73143322b78-fb1e7804-9542-4412-be66-e143a10e3b1a",
  "roleId": "6e4bf58a-b8e1-4cc3-bbf9-d73143322b78",
  "principalId": "fb1e7804-9542-4412-be66-e143a10e3b1a"
}
Enter fullscreen mode Exit fullscreen mode

Insomnia Screenshot:
Alt Text

Conclusion:

In this Post we just saw some cool ways to manage the synapse workspace purely using APIs. This opens-up the wide space for automation. Please keep in the mind that most of these features are in preview.So there is no such grantee that all the API endpoint/operations will be the same as now which i explained in the post.I keep my best level to update this post whenever some update needs.Here I explained one of operation like synapse role assignment for data plane.Actually Data Plane API can do much more than this.Read this github repo for getting the full power of Data Plane API Operations.

Top comments (0)

Read next

haamid profile image

Run Gitlab CI jobs on a specific runner

Hamid Haghdoost -

makowskid profile image

Introducing the Resume Parser/CV Parser for Laravel

Dawid Makowski -

ikelechia profile image

How to connect a Linux server via a Bash script.

Ikechukwu Kelechi Alexander -

sonmusui profile image

Unleashing the Power of AI: Running Large Language Models on Your Own Cloud Server (Digital Ocean)

Sonam Choeda -