Korean Food classification

Executive Summary

This project focuses on developing an application designed to detect, segment, and classify various Korean foods present on hospital meal plates. The solution aims to provide a streamlined, efficient method for analyzing meal contents, potentially aiding dietary assessments and nutritional analysis in healthcare settings.

The application leverages advanced computer vision and machine learning models to achieve accurate food detection, segmentation, and classification. Using YOLO for object detection, SAM (Segment Anything Model) for segmentation, and CLIP with FAISS indexing for classification, the system is structured to handle the diverse and complex appearances of Korean dishes. Additionally, optimizations were applied to enhance processing speed and minimize latency, making the app both effective and user-friendly in real-time or near-real-time applications.

The project encountered specific challenges, including ensuring high accuracy in food detection amidst visually similar dishes, achieving clear segmentation results, and maintaining a low response time. By addressing these issues through model tuning and parallel processing strategies, the app is now well-positioned to deliver robust results in a healthcare environment.

This solution represents a significant step forward in the use of AI-driven tools for food detection and classification within specialized dietary contexts.

Introduction

In hospital settings, accurate dietary monitoring plays a critical role in patient care and recovery. This project focuses on developing an application that can automatically detect, segment, and classify Korean foods present on hospital meal plates. By leveraging advanced computer vision and machine learning techniques, the application provides a streamlined solution for food recognition, supporting hospital staff and dietitians in dietary assessments and nutritional management.

Korean cuisine presents unique challenges for image-based food recognition due to the wide variety of dishes and visually similar ingredients. This application addresses these challenges by implementing a pipeline that combines object detection, segmentation, and classification models. With this tool, healthcare providers gain a reliable method to analyze meal components, potentially improving dietary planning and nutritional analysis within hospital environments.