I recently embarked on a fascinating journey inspired by Nicholas Renotte's tutorial on building a Facial Recognition Model from a research paper to code. Despite its complexity, the tutorial offered clear guidance, yet I encountered a few hurdles, particularly with Google Colab's integration with webcams.
Within the 8 part series we had to create sample images of our face using the web camera. Initially, I struggled with accessing my webcam in Google Colab. Nicholas was using OpenCV and that was not working via Google Colab. It wasn't until I discovered the hidden gem of code snippets that I realized there was a solution at my fingertips.
Lesson #1: Google Colab offers invaluable resources through its code snippets feature.
Forgive me if you already knew this but I didn't so that counts as a lesson.
However, my challenges didn't end there. I needed to capture hundreds of facial images, which proved to be a tedious task with the default camera capture snippet. Enter
Lesson #2. Google Colab's ability to execute JavaScript within the notebook.
This newfound knowledge revolutionized my approach. I customized the camera capture process. I replaced closing the camera after the picture was taken to a button-driven image capture that closed on command. Harnessing JavaScript's power, I extended functionality, maintaining the camera connection until my task was complete.
All my focus has been on Data Science lately and not one time until now did I realize that you can run Javascript within the notebook. Honestly I never had to, but that's not the point.
Yet, even armed with JavaScript, I encountered my final obstacle: integrating Python functions with JavaScript callbacks. It took some troubleshooting and a return to the Google Colab documentation to pinpoint the missing link. With
Lesson #3. Orchestrating a seamless interaction between JavaScript and Python.
Turns out not only does Javascript have to call the python function.
// Send the base64-encoded image data to Python
const is_saved = await google.colab.kernel.invokeFunction('notebook.save_image', [imageData, full_file_path], {});
But you also have to register the call back.
output.register_callback('notebook.save_image', save_image)
I mean like, duh. Be patient with me I'm being vulnerable here.
The culmination of these lessons resulted in a robust workflow, enabling the capture and processing of over 400 facial images for model training. Witnessing the model in action was immensely gratifying, and I'm eager to share my journey with others, hoping to spare them similar trials. For those interested in exploring my progress or delving into Nicholas Renotte's invaluable tutorials, I've shared my work on GitHub and encourage following Nicholas for further inspiration.
Check out my progress on GitHub: GitHub
And don't forget to follow Nicholas Renotte: Nicholas Renotte's YouTube Channel
Oh and here's are my code snippets.
crops the images
def crop_square(img, size, interpolation=cv2.INTER_AREA):
h, w = img.shape[:2]
min_size = min(h, w)
# Centralize and crop
center_x, center_y = w // 2, h // 2
half_size = min_size // 2
crop_img = img[center_y - half_size:center_y + half_size, center_x - half_size:center_x + half_size]
resized = cv2.resize(crop_img, (size, size), interpolation=interpolation)
return resized
saves the images
def save_image(image_data, full_file_path):
try:
# Decode the base64-encoded image data
binary = base64.b64decode(image_data)
# Load the image directly as a NumPy array
image_array = cv2.imdecode(np.frombuffer(binary, dtype=np.uint8), cv2.IMREAD_COLOR)
# Process the image
processed_image = crop_square(image_array, 250)
# Save the processed image to a file
cv2.imwrite(full_file_path, processed_image)
return True
except Exception as e:
print(f'Error saving image: {e}')
return False
output.register_callback('notebook.save_image', save_image)
opens the webcam for some clickety click action
def take_photo(dir, generate_file_name=True, quality=0.8, verification_callback=None, capture_btn_name='Capture'):
if generate_file_name:
# name the new file with unique identifier
full_file_path = os.path.join(dir, f"{uuid.uuid1()}.jpg")
else:
full_file_path = dir
# create javascript function
js = Javascript('''
async function takePhoto(full_file_path, quality, verification_callback, capture_btn_name) {
// create shell with div and capture button
const div = document.createElement('div');
div.style.padding = '10px';
// create capture button
const capture = document.createElement('button');
capture.textContent = capture_btn_name;
capture.style.marginRight = '10px';
div.appendChild(capture);
// create close button
const close = document.createElement('button')
close.textContent = 'Close';
div.appendChild(close);
// create video space
const video = document.createElement('video');
video.style.display = 'block';
// add video block to div
div.appendChild(video);
// create video element
const stream = await navigator.mediaDevices.getUserMedia({video: true});
// add dive to the body
document.body.appendChild(div);
// set the source and play the media
video.srcObject = stream;
await video.play();
// resize the output to fit the video element.
google.colab.output.setIframeHeight(document.documentElement.scrollHeight, true);
capture.onclick = async () => {
// create a canvas element to draw the video frame
const canvas = document.createElement('canvas');
// set the canvas width and height to match the video dimensions
canvas.width = video.videoWidth;
canvas.height = video.videoHeight;
// draw the current video frame onto the canvas
canvas.getContext('2d').drawImage(video, 0, 0);
// Convert the canvas content to a base64-encoded JPEG image data URL
// with the specified quality
const dataUrl = canvas.toDataURL('image/jpeg', quality);
// Split the data URL to extract the base64-encoded image data
const imageData = dataUrl.split(',')[1];
// Send the base64-encoded image data to Python
const is_saved = await google.colab.kernel.invokeFunction('notebook.save_image', [imageData, full_file_path], {});
// If the verification callback is provided, invoke it
if(is_saved.data && verification_callback !== undefined){
const result = await google.colab.kernel.invokeFunction(verification_callback, [], {});
console.log(result);
}
return is_saved.data;
};
close.onclick = () => {
// stop the video stream to release resources
stream.getVideoTracks()[0].stop();
// remove the video element and its containing div from the DOM
div.remove();
};
}
''')
# create and display the javascript function
display(js)
# call the javascript function
display(Javascript(f'takePhoto("{full_file_path}", "{quality}", "{verification_callback}", "{capture_btn_name}")'))
Top comments (0)