Azure Data Factory is perhaps the most loved one-stop data orchestration platform that is available out there, And it is essentially very important to use ADF along with source control if you are going to deploy it to multiple environments then without having automation it can be very hard to track changes deploy, etc,
Here I am following the below method depicted in the image.
So the first thing that I have done is create two Resource groups one for DEV and another for the TEST environment.
Two dedicated resource groups for two environments :
Each of the Resource groups contains 1 ADF and 1 Blob :
Now log in to the ADF from the dev environment and connect it to the source control either Azure repo or Github,
Once connected, try to publish your workspace, if you have default settings then an additional *adf_publish *branch will be created which will hold all the ARM templates required to deploy the ADF.
If you look at the branching strategy we are keeping only the DEV environment under source control all the other environments will not be connected to any repository.
Now if you open the adf_publish branch after successful deployment you may see the brach got populated with all the templates,
Since we have to deploy this same thing to other environments let us design the release pipeline.
In the Azure DevOps Release pipeline, the Artifactory location is given as the adf_publish branch.
Now, The next stage is configured which will apply the updated changes to the TEST environment
if you look while doing these hands-on you must change the parameters by using the override parameters section available in the ARM template deployment task,
what it will do is change the blob connection string to the test environment blob connection string which is a necessary step.
Pro TIP :
Rather than using override parameters, you can use the PowerShell/Bash command to override the string,
for example: (Get-Content -Path .\test. txt) -replace ‘string_one’,’string_two| Set-Content -Path .\test. txt
Let’s see everything in action:
In the dev environment ADF, we create a new branch based on the main branch,
Points to note Publish is not possible from this feature branch, Once work is done and validated we will apply the same and create a pull request,
Once the merger check passes the approver can merge the pull request,
Once it is successfully merged we can delete the branch
Now the updated feature is in the main branch, So we go to the main branch and publish all the changes to VCS, We will get this kind of notification showing to publish is successful,
Now if we just go to the Azure DevOps and hit create a release, It will just start creating the updated features and resources in the TEST environment from the DEV environment,
after the release pipeline successfully completes you can see changes made in the dev branch reflected in the release branch as well.
Top comments (0)