Where Should I Put My Data in Azure

Let's start by moving those images out of your database. That was my first recommendation, most of the time, when meeting with clients to help them achieve better performance. That was a little while ago; it was simple. These days it's easier to get confused with all the options we have to save our data.

In this post, I want to share some highlights shared with me recently during a discussion with the Product Manager of Azure Data Lake, Azure Data Explorer, Cosmo DB, and Azure Synapse Analytics.

Type of data

There are multiple types of data: transactions, logs, histories, and files of all types from csv to json. All of that information can be structured or unstructured. Even more, sometimes that structure (referenced as schema) often changes.

Different Tools for Different Scenarios

This is why there are different types of tools to help query your data more efficiently depending on what it is. Of course, the quantity and the type of queries that you would like to run also influence the tool you would select. Let's see a few of them.

Azure Data Lake

Azure Data Lake is a file system meant to store an amazing quantity of data that will support scale at a very affordable cost. Azure Data Lake is in fact a set of capabilities on top of Azure Storage and would be perfect for mostly any type of data: IoT data, sales data, logs that we would like to analyze after.

In a recent interview I did with Jeff King, you can hear him talking about some do's and don'ts with Azure Data Lake. It was really interesting listening to all his recommendations and about The Hitchhiker's Guide to the Data Lake, a must for all the current users of Azure Data Lake or if you are planning a migration.

Get started with this great learning module: Introduction to Azure Data Lake Storage available on Microsoft Learn.

Azure Data Explorer

Azure Data Explorer is an analytics database, so it's not a common transactional database used to store your data but instead its for analyzing the data. It excels in time series and searching text in unstructured or structured data. It's extremely fast at returning query results thanks to an automatic compression system.

Vincent-Philippe Lauzon answered all my questions about Azure Data Explorer, Kusto, and showed me how this tool works during this short interviewing on Hello World.

Vincent also mentioned a learn module to get started Introduction to Azure Data Explorer. A create way to learn to describe the ingestion, query, visualization, and data management features that Azure Data Explorer provides to help you make sense of the data flowing into your business.

Azure Cosmo DB

Cosmo DB is different than other databases because it is a horizontally scalable database, which gives it mostly unlimited storage capacity and great performance. Perfect to be used in large ingestion of data or when you need high availability around the globe. In my interview with Mark Brown, he explained those differences and shared some scenarios where Azure Cosmo DB will definitely shine.

There is an entire learning path available on Microsoft Learn named Work with NoSQL data in Azure Cosmos DB. It is the perfect way to learn about NoSQL, great tools and get started with Cosmo DB.

Azure Synapse Analytics

Azure Synapse Analytics is all about doing enterprise analytics at scale. What's really great about Synapse is that it works with all of the other platforms, unifying the tools and the teams. Like Saveen Reddy was saying when we talked with him: Azure Synapse is about erasing limits between teams and removing the boundaries. He also shared scenarios where Synapse is at its best. You can watch this interview here.

There is also Introduction to Azure Synapse Analytics, a great module to learn the features and components that Azure Synapse Analytics provides. A one-stop shop for all your analytical needs.

Wrapping Up

Having the chance to have all those great people reunited was wonderful. I strongly suggest having a look at Hello World episode Where Should I put my Data in Azure. After each interview or demo, we did a period of Q&A (that's how we like it on the show), and thankful that Amy Boyd was co-hosting with me, because there were many questions. Stay tuned as I think I will do a second episode on this theme...

Starting to left Jeff King, Mark Brown, Vincent-Philippe Lauzon, and Saveen Reddy

☁️