Azure Data Factory Mainly Follow Two Approaches
- Direct Copy Approach .
- Stage Copy Approach .
- Direct Copy Approach
To load Data From Sources like Azure Blob Storage or other Azure Storage Services and Destination Other than Snowflake, In that case we can use Direct copy Approach , We do not need to Configure Staging area . By Just passing Source Data Set and Sink Data Set we can Implement our Pipeline .
- Stage Copy Approach
If We Want to Load our Data to Snowflake from Sources other than Azure Storage Services , we will use a Stage Copy Approach. For that we need to Configure our staging , and pass the Source Data Set as well as Sink Data Set .
Now Let's See how we are Loading data from MYSQL to Snowflake using ADF
*Steps - *
- Setup Azure Account → SignIn into your account → Go to Azure Portal.
- In Azure select and create a new Project in Azure Data Factory.
- Setup Snowflake Account → Create a Users → Create a Database and Schemas.
- In Studio Go to the Manage Section , And Create a new Linked Service To Connect MYSQL with ADF.
- After that , Create a New Linked Service for Snowflake .
- Now go to the author section in ADF and click on the icon of the data set and create a new data set from the source table.
- After creating data sets from source, now create a data set for destination. For this we need a table in our snowflake account, so create a table in a snowflake account and then in ADF create a new data set for the snowflake table.
- After you are done with creating data sets now we are good to move to create a pipeline. For this you need to go in the pipeline section and create a new pipeline, In activity section you will Copy Data.
- Now , click to the Copy Data command and configure the Source and Sink Tabs.
- Run your Azure Data Factory Pipeline to load the data into Snowflake. You should see in Snowflake a call to the ODBC in your history table. This is how you know ADF has successfully landed data in Snowflake .
Top comments (0)