Let's continue my recent trend in exploring pricing tips for the various parts of AI and Machine Learning on Azure with a dive into Azure Cognitive Search.
Sometimes confused with the AI offerings of Azure Cognitive Services, the entirely different Azure Cognitive Search is a rich service that allows you to index a variety of files and documents, extract meaning from those documents, and provide rich search results to users.
In this article we'll explore the pricing structure of Azure Cognitive Search and highlight some things you should be aware of as you plan and develop your Cognitive Search resources.
Let's start with my standard disclaimer on pricing: this article is based on my current understanding of Azure pricing, particularly in the North American regions. It is possible the information I provide is incorrect, not relevant to your region, or outdated by the time you read this. While I make every effort to provide accurate information, I provide no guarantee or assume no liability for any costs you incur. I strongly encourage you to read Microsoft's official pricing information before making pricing decisions.
Azure Cognitive Search Pricing
With Azure Cognitive Search, your costs come from several different channels:
- Per scale unit (SU) costs for the actual searching on Azure Cognitive Search
- Additional costs for premium features such as semantic search, image extraction, and custom entity lookup
- Storage costs for the stored content
The base cost for the Azure Cognitive Search entry depends on which edition you selected of Azure Cognitive Search.
Editions
Like an App Service or similar services, you can choose what tier of Azure Cognitive Search you want.
Note: this data is included for reference purposes to show you the types of differences you see between editions. Pricing and edition breakdowns are likely to be different by the time you read this, particularly if you use a region other than Central US. Refer to the official pricing documentation for accurate information.
Azure Cognitive Service costs range from a free tier to paid to more and more expensive editions as you move into the more powerful editions. Also note that if you have a certain amount of data you need to store or certain number of indexes you require that you may be forced into more expensive tiers.
This leads us to our first point: plan your indexed data sources carefully. In my experience, the quantity of disparate sources of things you index typically pushes you towards the higher end plans before the storage needs do.
If you have an option between searching 5 different sources of information or doing some extra work to consolidate those 5 sources to a single source, I'd recommend you do the consolidation, even if it adds more complexity to your workflow, because a single source tends to price significantly better in Azure Cognitive Search than any number of different sources that may grow over time.
This consolidation is going to look different based on your application's architecture, but it could mean using a single Storage account to store disparate types of information, or it could involve some sort of manual or automated process to collect things from different systems and stage them in a single location where Cognitive Search can find them.
If you're anything like me, you'll be drawn to the free tier initially during development. This is a great tier to be on as you prototype your applications, but an awful one to be on for production applications.
If you want to use the free tier during development, keep in mind that you only get a single free tier of Azure Cognitive Search instance on your account before you're forced into the paid plans, so use it wisely!
Storage Costs
The information you store on Azure for indexing is still subject to storage costs. For example, if you index a storage account, you still must pay for the items that are stored depending on their usage frequency and whether the storage is hot or cold.
Relational databases would incur a similar and steeper charges, however it is likely you are using this database for other applications and not just as a way of storing data to be indexed.
Keep in mind that secondary charges such as Security Center or backups may also apply depending on the service that stores your data and your account's policies, and that storage costs generally only grow over time as volume of data stored increases.
Premium Capabilities
There are a few premium tier capabilities that you can add to search indexing that increase your rates beyond Azure Cognitive Search's core costs.
If you define your own custom entity lookup skill, you will pay an additional cost during indexing based on the quantity of records indexed with this skill. The rates are on par with the fraction of a penny per row types of rates you'd see from Azure Cognitive Services, but obviously the larger the dataset you have the more this per-row fee adds up.
Additionally, if you apply document cracking to image-based documents you will pay an additional fee as well.
This document cracking image capability is often a key contributor for people using Azure Cognitive Search in the first place, so it's less of a fine to be avoided at that point and more of the base cost of the service.
If you find this cost prohibitive, you could try using some compute resources outside of Azure and a library like OpenCV or Azure Form Recognizer to try to extract meaning from image documents and then store the extracted information in Azure Cognitive Search instead of the source document. This typically is not worth the effort, however.
Finally, you can opt-in to semantic search to improve the results of searches by also considering search terms that are semantically identical to the terms you searched for. This is free for paid Cognitive Search plans using low-volume scenarios (fewer than 1,000 queries per month) but costs an additional fee based on excess usage depending on the edition of Cognitive Search you're using.
Semantic search usually improves the effectiveness of search significantly, so much that it can be hard to disable to save money once your users are used to the improved results.
Conclusion
This concludes the exploration of pricing structures and tips in Azure Cognitive Search.
Azure Cognitive Search is an area of development I haven't done as much work in as other Azure AI / ML offerings, but it is a powerful and cost-effective solution for indexing a variety of sources of information and adding additional context to the search results in the form of extensible skills.
As with anything on Azure, I encourage you to calculate your costs up front with the pricing calculator and rely on Azure Cost Management, Budgets, and Alerts to show you how your actual costs are stacking up against your estimates.
Stay tuned for the final pieces of content as my Azure Spring Clean 2023 series concludes by covering OpenAI on Azure. Make sure you check out my prior pieces on Azure Machine Learning, Azure Cognitive Services, and Azure Bot Service.
Top comments (0)