The Challenge
Manual Knowledge Graph creation is a massive bottleneck. Existing pipelines rely heavily on text-to-graph extraction, completely ignoring the spatial logic hiding inside visual object detection data required for industrial robotics.
Bachelor Thesis
Automated knowledge graph creation via multi-object detection for an AI maintenance assistant
- Python
- owlready2
- RDF / OWL
- DeepSeek / Llama / Qwen
- SSD Object Detection
- Git
- 1.4
- Final Grade
- 4
- LLMs Evaluated
The Approach
I built a Python pipeline that ingests bounding box coordinates, applies geometric algorithms to extract spatial relations ('left_of', 'inside_of'), and compiles them into standard-compliant OWL ontologies. I also implemented multi-camera fusion to map complex 3D relationships from 2D feeds.
The Impact
Proven that deterministic geometric-to-semantic translation can match or beat massive multi-modal end-to-end models in industrial contexts. The defense and paper earned a 1.4 grade for strict methodological rigor and LLM evaluation.
System Architecture
The system processes annotated image data (CSV) containing bounding boxes. It calculates the center of each object and applies threshold-based logic to determine spatial relationships. It utilizes `owlready2` to generate standard-compliant RDF triples. The output was evaluated against four LLMs (DeepSeek-R1, DeepSeek-V3, Llama 3.1, Qwen 2.5) using specific metrics for correctness and completeness.
Key Engineering Features
Geometric Relation Extraction
Algorithms to automatically determine 'above', 'below', 'left_to', and 'inside_of' relations based on pixel coordinates.
Multi-Camera Fusion
Logic to merge object detection data from multiple angles into a single consistent Knowledge Graph.
Implicit vs. Explicit Modeling
Implemented variants to test graph compactness versus semantic completeness for LLM processing.
LLM Integration
Evaluated different serialization formats (OWL vs. Triples) to optimize LLM reasoning capabilities.
Development Lifecycle
Research & Concept
Analyzing state of the art in Semantic Web and Object Detection.
Implementation
Developing the Python generator and relation logic algorithms.
Submission
Final evaluation of LLM performance and thesis submission at TU Dresden.
Defense
Final defense of thesis at TU Dresden. Received a grade of 1.4