AI & methodology
From a question to a face-to-face answer.
A deep learning, multimodal, generative AI system. The technical pipeline is being designed during the Investigate phase — this page grows with it.
Deep learning, multimodal
Voice, visuals and language combined in one system.
Generative AI
The avatars generate their answers to students’ questions.
3D avatars with a voice
Professors, thinkers and great minds that speak and teach.
In the student’s language
Answers in the student’s own language, face to face.
The pipeline
Step by step, how an avatar answers.
Input
The student's question
Any student, wherever they are, asks a question — in their own language.
Step 1
Speech recognition
Understands the spoken question — Derja, French or both, even in noise. Budget: ~100 ms.
Candidates: Whisper · audio-visual transformer
Step 2
Course-grounded answer (RAG)
Searches the professor’s own course, answers with the cited source, guides with hints first — or says “not in your course”. Budget: ~200 ms to the first token.
Candidates: LLM + RAG · ColPali · VLM
Step 3
Voice synthesis
Speaks the answer with the consented, cloned voice of the professor, in the learner’s language. Budget: ~100 ms to the first audio.
Candidates: TTS · Seamless · UniSS
Step 4
Face animation & lip-sync
Animates the avatar’s face and syncs its lips to the voice, in real time. Budget: ~100 ms with the network.
Candidates: Wav2Lip · viseme network · diffusion
Output
A 3D avatar answers, face to face
A professor, thinker or great mind answers with a voice, as a 3D avatar.
Under the hood
Models, datasets & metrics.
Models
Whisper · AV-Transformer
Module 01 — Voice & Audio — speech recognition that also reads lip movement in noise
LLM + RAG · ColPali · CLIP / VLM
Module 02 — Language & Knowledge — sourced answers from courses, slides and photos
Wav2Lip · viseme network · diffusion
Module 03 — Face & Video — realistic face and lip-sync
Seamless · UniSS · NLLB
Module 04 — Multilingual — translation that keeps the original voice
Deep Knowledge Tracing · LLM
Module 05 — Adaptive Personalization — tracks mastery from the conversation, never from the face
LLM · text-to-image · TTS
Module 06 — Content Generation — narrated lessons, diagrams, summaries and quizzes
Datasets
Read, annotated voices in 100+ languages (CC0) — weak for Tunisian dialect
FLEURS
Read speech in ~102 languages, ~12 h each — multilingual ASR evaluation
MGB-3/5 · TARIC · MADAR
Egyptian and Tunisian Arabic speech — on request, research licences
Open textbooks and ~2,500 university courses — test content for RAG
FLORES-200 · OPUS
Translation in 200 languages — few dialects
VoxCeleb · HDTF · MEAD
Talking faces — research only, never in the product without a licence
Metrics
End-to-end conversational latency
From the end of the question to the avatar’s first word (PP2, SG3)
Target < 500 ms
Lip-sync and frame rate
SyncNet score and FPS on our hardware (EX2)
Target ≤ 30 ms · 200+ FPS
Sourced answers
% of answers citing the course; % correct “I’m not sure” (EX3)
Target sourced answers
Voice cloning
Similarity, naturalness rated by 10 students, inference time (EX1)
Target < 120 ms
Teaching value
Quiz score, engagement and satisfaction: video vs interactive avatar (EX7)
Target A/B test
Usability
System Usability Scale with 5 students and 2 professors (PP4)
Target SUS score
Responsible AI
Transparent by design.
Principles set during the Engage phase, which every technical choice will follow.
Clear disclosure
AI-generated avatars and voices are always labelled as such, as required by the EU AI Act since August 2026.
Voice-cloning awareness
A voice can be cloned from 3 seconds of audio — voices are handled with care and transparency.
Building trust
Only 46% of people trust AI while 66% use it: transparency is our difference.
AI & Methodology
Coming soon.
Our technical pipeline is being designed during the Investigate phase. This page will be unveiled as soon as it is ready.
Follow our CBL journey