Problem
Maintenance instructions are usually separated from the machine they describe. The prototype explores whether detected components and spatial guidance can be placed directly in the technician's field of view instead.
Back to Projects
Research project · TU Dresden · 2024 - 2025
AI Maintenance Assistant
A projection-based maintenance prototype that detects machine components on a server and maps visual guidance back onto the physical workspace.
- Python
- TensorFlow
- Docker
- Flask
- Raspberry Pi
- SSD Object Detection
- Git
- AR Projection
- Platform
- Raspberry Pi 3
- Hardware
A distributed hardware-software prototype for industrial maintenance. A Raspberry Pi captures and validates camera frames, discovers the inference server on the local network, and projects returned component overlays. The Dockerized Flask backend runs a custom SSD detector and converts its output into spatial machine representations.
Approach
Split the workload between a Raspberry Pi and a server. The edge device handles camera capture, brightness and blur checks, calibration, server discovery, and full-screen projection; the server exposes Flask endpoints for object detection and generates component-level overlay instructions.
Outcome
Produced an end-to-end research prototype that connects edge hardware, containerized inference, spatial data generation, and projector output. The same pipeline later became the basis for research into automatically generated knowledge graphs from object detections.
Architecture
A Python client-server system connected over the local network. The Raspberry Pi sends base64-encoded camera frames to a Flask API and receives detection coordinates as structured data. The server delegates inference to a Dockerized SSD model, derives spatial relations and overlay instructions, and returns them to the edge device for projection.
Highlights
Object Detection
Fine-tuned SSD object detection on a custom dataset of industrial machine components.
Edge Hardware
Raspberry Pi camera capture, calibration, image-quality checks, and projector output.
Containerization
Dockerized architecture for consistent deployment across edge and server environments.
Network Discovery
Automatic discovery and health checks for the inference server on the local network.
Timeline
Model Optimization
Fine-tuned SSD object detection on a custom dataset of maintenance tasks.
Edge Coordination
Developed the Python backend for both the server and the edge device.
Feature Expansion
Added image validation, camera calibration, network-wide server scanning, and the projector-based AR overlay.
Knowledge Graph Extension
Extended the detection output with geometric spatial relations and multi-camera knowledge graph generation.