Challenging the AWS AI Practitioner Beta - My exam experience and insights

#aws #certification #ai

One of the new AWS exams is the AWS AI Practitioner beta. I share my experience with this exam, covering prep, key topics, and insights on its challenging nature.

1. Background

AWS recently announced two new certification exams: the AI Practitioner Foundational and the Machine Learning Engineer Associate. As of this writing, both exams are in beta and will remain in this phase for a few more weeks while AWS gathers sufficient data and feedback.

The first available date for the beta exams was August 27, 2024, so I scheduled my AI Practitioner exam for that day.

In this post, I’ll share my insights on the preparation process and the exam.

2. Preparation

I already hold the Machine Learning Specialty certification. Since AWS categorizes the AI Practitioner exam as foundational, I anticipated I could quickly fill any knowledge gaps and be ready for the exam.

To prepare, I followed the Enhanced Exam Prep Plan: AWS Certified AI Practitioner course on AWS Skill Builder. With a paid monthly subscription, I had access to managed labs and the full-length official practice exam.

I completed all the recommended courses, finished the labs, and wrapped up the Exam Prep Enhanced Course. Based on my performance in the practice question sets, I felt confident going into the exam.

3. The exam

As a beta exam, the AI Practitioner consists of 85 questions instead of the usual 65. The allocated time is 120 minutes, with an additional 30 minutes if English is not your first language and you're taking the exam in English.

After the first few questions, I realized that the exam was more challenging than I had anticipated, both in terms of the required knowledge and the complexity of the questions. It felt more akin to an associate-level exam rather than a foundational one.

3.1. Concepts and services

I anticipated that the exam would include many questions about generative AI and Amazon Bedrock, and I was correct. Be prepared to understand Bedrock's features and have a solid grasp of general generative AI concepts.

There were also numerous questions on responsible AI, focusing on risks, biases, and how to address or prevent these issues. Familiarity with prompt engineering techniques is essential.

A significant portion of the exam covered SageMaker. If you already hold the ML Specialty certification, you’ll find these topics familiar.

The exam also tests your knowledge of other managed AWS AI services, like Comprehend and Transcribe. These questions generally focused on identifying the appropriate service for a given scenario, which aligns more with the foundational category of the exam.

Interestingly, many questions did not involve specific AWS services at all. Instead, they assessed general ML and AI knowledge. Concepts such as classification metrics or the confusion matrix were present, but many questions also focused on generative AI.

Understanding the ML development lifecycle is crucial. I encountered several questions about deployments and inference options, which delved into a fair amount of detail.

As with other AWS certification exams, security is a significant component. If you’re familiar with AWS security services and features like IAM, Macie, or PrivateLink, these questions should not come as a surprise.

3.2. New question types? No!

With the introduction of the new exams, AWS also announced three new question types: ordering, matching, and case study.

I was particularly eager to encounter these new question types, especially the case study format. This type presents a scenario followed by two or more related questions. I find this approach beneficial as it reduces context switching — allowing you to focus on less scenarios. Additionally, it requires a deeper understanding of the topic since the questions examine the scenario from multiple angles.

To my surprise, I didn't encounter any of these new question types in my exam. I'm unsure whether this was simply a matter of chance or if AWS plans to introduce these questions at a later stage. I was disappointed when I reached the final, 85th question and realized that the majority were traditional multiple-choice, with only a few multiple-response (e.g., "select 2 out of 5") questions.

Another unexpected aspect was the wording of some questions, which I found ambiguous. This was a new experience for me, as I hadn't encountered such issues in previous AWS exams. It's possible that this was due to gaps in my knowledge on certain topics, or it could be a characteristic of beta exams (this was my first beta exam).

In my opinion, individuals new to the ML/AI domain will likely need more than the recommended six months of experience to feel confident about taking this exam.

3.3. After the exam

One positive aspect is that we don’t have to wait weeks to receive the results. AWS notifies candidates whether they passed or failed within five business days, which is the same time frame as for non-beta exams. In practice, this usually happens within 24 hours. I only had to wait about five hours before receiving the email confirming that I had passed the exam.

4. Summary

Overall, I found the exam to be challenging compared to its positioning as a foundational-level certification. I believe its difficulty is much closer to the associate level.

That said, the exam effectively tests your knowledge of ML and AI. If you’re considering taking this beta exam, make sure to study the concepts outlined in the exam guide thoroughly.