This content originally appeared on DEV Community and was authored by Malar Kondappan
In the previous section, we covered why emotion detection matters and how computers “see” feelings.
Now, let’s explore how to implement this in code, understand the architecture, and make sense of the workflow, with real examples and explanations for each step.
Project Architecture: The Three Pillars
Roboflow Dataset Manager:
Gathers and formats image data for training
GitHub RepositoryYOLOv11 Model Training:
Fine-tunes a neural net to recognize emotions
GitHub RepositoryFace Emotion Detection System:
Runs the model for real-time inference on Raspberry Pi
GitHub Repository
1. Preparing the Dataset (Data Science Foundation)
Before your AI can recognize emotions, it needs to learn from thousands of labeled examples.
Use Roboflow Universe to find or create emotion datasets.
The dataset manager script automates download and formatting in YOLOv11-compatible folders.
Sample Python code to download with Roboflow:
from roboflow import Roboflow
rf = Roboflow(api_key="YOUR_API_KEY")
project = rf.workspace("your-workspace").project("your-emotion-project")
dataset = project.version(3).download("yolov11") # Public datasets may skip API key
# Output: Folders with images and YOLOv11 labels (train/valid/test subfolders)
# Each image gets a companion .txt file with bounding boxes and class labels for the detected emotion.
2. Training the Model (Machine Learning in Action)
Now, let’s teach YOLOv11 to spot emotions. The magic happens through transfer learning, building on a pre-trained model (Nano version) and specializing on emotion data.
Install dependencies:
pip install ultralytics torch opencv-python
Start training (Python):
from ultralytics import YOLO
# Path to dataset config (created above)
model = YOLO('yolo11n.pt') # Nano base model
results = model.train(data='data.yaml', epochs=200, imgsz=320, batch=10, device='cpu', augment=True)
-
data.yamldescribes paths and classes -
epochs=200: More epochs, better accuracy (but risk overfitting) -
imgsz=320: Smaller images = faster for Pi, good enough accuracy -
batch=10: Number of images processed per step -
device='cpu': Set to'0'for GPU if available
Inspect results:
# After training
model = YOLO('runs/detect/train/weights/best.pt') # Best model selected automatically
img = 'test-image.jpg'
results = model(img)
results.show() # Visualize detections and emotion labels
Best weights (best.pt) are saved for deployment; model size just 6.5 MB!
3. Deploying for Real-Time Raspberry Pi Inference
Now the excitement, deploy your model to Pi and see instant results!
Essential script highlights:
import cv2
from ultralytics import YOLO
model = YOLO('yolo-trained-models/emotionsbest.pt')
cam = cv2.VideoCapture(0) # 0 for Pi Camera; use correct ID for USB webcam
while True:
ret, frame = cam.read()
results = model(frame)
for r in results:
boxes = r.boxes.xyxy
labels = r.names[r.probs.argmax()]
conf = r.probs.max()
# Draw bounding box and label on frame
# Color-coding for each emotion: see STANDARDCOLORS dict in app-pt.py
cv2.imshow('Emotion Detection', frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cam.release()
cv2.destroyAllWindows()
Performance tips:
- Reduce image resolution for faster speed:
picam2.preview_configuration.main.size = (1920, 1080) - Skip frames if slow (
N = 2means process every 2nd frame) - The script auto-detects Pi Camera and USB webcams
Code Architecture Explained
- Input: Frames from Pi camera
-
Processing Pipeline:
- Convert color spaces (BGR ↔ RGB as needed)
- Feed frame into YOLOv11 model
- Get boxes, class labels, confidence scores
- Annotate output image with emotion predictions
- Display live feed (with FPS metrics)
-
Customization:
- Edit emotion colors (
STANDARDCOLORS) - Adjust camera resolution and frame skipping (
N) - Record video or save frames with OpenCV’s
VideoWriter
- Edit emotion colors (
Troubleshooting & Optimization (Tips for Developers)
Camera not detected?
libcamera-hello --list-cameras
sudo raspi-config # Enable Pi Camera module
Model too slow? Lower imgsz, increase N, use GPU if available
Out of memory? Lower batch size, image size
Accuracy not improving? Check results charts, confusion matrix images; retrain with more data or longer epochs
Advanced Techniques
- Fine-tune with your own emotions: Update the Roboflow script to download/label custom datasets, retrain as above.
- Export to ONNX: For deploying to other platforms:
model.export(format='onnx')
- Integrate with applications: Use inference results to log emotion metrics, trigger events (robotic reactions, game states, feedback systems).
Example: Full Inference Script (Python)
Here’s how a simplified detection loop looks:
from ultralytics import YOLO
import cv2
model = YOLO('emotionsbest.pt')
cam = cv2.VideoCapture(0)
while True:
ret, frame = cam.read()
results = model(frame)
for result in results:
for box in result.boxes:
x1, y1, x2, y2 = map(int, box.xyxy)
emotion = result.names[box.cls]
conf = box.conf.item()
# Draw box and label code here
cv2.imshow('Live Emotion Detection', frame)
if cv2.waitKey(1) == ord('q'):
break
cam.release()
cv2.destroyAllWindows()
Add your preferred annotation, color coding, and FPS monitoring!
Takeaway
- The journey from raw images to real-time emotion detection combines data science, machine learning, and systems engineering.
- Each code block is an invitation to experiment, learn, and innovate.
- Good luck building and let’s make edge AI more expressive, accessible, and creative!
Get Hands-On: Ready-to-Experiment Project & Community Invitation
Interested in building a complete, working facial emotion detection system?
This open-source project is designed for you to experiment, learn, and contribute; whether you’re a beginner, teacher, student, or developer.
What you can do with this project:
- Set up a live emotion recognition pipeline using real Raspberry Pi code and datasets.
- Tweak and train the model for your unique classroom, hobby, or research needs.
- Open issues, give feedback, or suggest new features, everyone’s ideas are valued.
- Fork, modify, or contribute your own improvements. Pull requests welcome!
Want to start right now? Visit the repos for everything needed, data management, training, and deployment:
Every star, fork, suggestion, or classroom trial helps grow this project and makes the technology more accessible for everyone.
Questions, ideas, or feedback? Comment below or reach out via GitHub! Let’s learn and create together.
This content originally appeared on DEV Community and was authored by Malar Kondappan
Malar Kondappan | Sciencx (2025-11-24T04:09:50+00:00) Deep Dive: Building Real-Time Facial Emotion Detection on Raspberry Pi with YOLOv11. Retrieved from https://www.scien.cx/2025/11/24/deep-dive-building-real-time-facial-emotion-detection-on-raspberry-pi-with-yolov11/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.