Hands-free AI Study Coaching Service using Meta Ray-Ban Smart Glasses
- MIT Reality Hack 2026 - Grand Prize (Gold Award) Winner
- MIT Reality Hack 2026 - Meta Track Winner
- Scheduled to attend AWE USA 2026 (World's #1 XR + AI Event)
- Goal: To eliminate the hassle of manual study logging and develop an AI coach that automatically analyzes and provide personalized guided study coaching using the 1st-person perspective (POV) of Meta Ray-Ban glasses and a companion app.
- Core Value: Visualizing the learning journey without ever interrupting learning itself, transforming fragmented study habits into a unified, actionable "study story" that delivers personalized post-session coaching that tells students not just what they covered, but also what to do next. .
- Manual Logging Hassle: Requirements for manual logging (starting timers, selecting subjects) break the flow and cause distractions.
- Lacks Contextual Data: Simply measuring 'study time' lacks the contextual feedback needed to understand what was actually covered.
- Low Output vs. Input: A lack of feedback loops means the energy spent on logging doesn't effectively lead to improved learning efficiency.
- Hands-free Tracking: Meta Ray-Ban cameras and microphones quietly observe and automatically log study sessions as they happen.
- Contextual Insight: AI analyzes 1st-person POV images to identify study content, engagement levels, and distractions (like phone usage).
- Customized AI Pipeline: Primary data is further processed via our customized AI Pipeline for providing coaching items such as flashcards and quizzes.
- Automatically tracks study sessions, subjects, and time spent without needing to touch a phone.
- AI automatically tags subjects (Biology, Math, etc.) and subtopics from the 1st-person POV images.
- Learning State Analysis: Determines if the student is actively solving problems or passively reading.
- Study Telemetry: Provides consistent, machine-readable metrics on learning quality using structured JSON data.
- Distraction Monitoring: Detects distractions, such as central phone visibility, within the user's field of view.
- In-session Voice Support: If you're "Stuck? Just ask" — receive instant voice-based guidance without breaking focus. Our principle is designed to avoid giving direct answers, instead asking reflective questions to encourage reasoning.
- Post-session Flashcards & Quizzes: Glasses capture and AI backend generate flashcards and quizzes automatically without mannual work for distaction-free study coaching.
- Visualizes metrics such as time per topic, active/passive ratios, and help counts via an iOS application.
| Category | Tech Stack |
|---|---|
| Frontend | Swift / SwiftUI (iOS App), Xcode |
| AI Models | GPT-4o (Vision), GPT Realtime API |
| Backend | Node.js, PrismaDB, WebSockets |
| Infrastructure | AWS S3, HTTP API |
| Hardware | Meta Ray-Ban Smart Glasses |
Hyunseok Hwang
Product Engineer |
Sylvan (Xiaofan) Shen
Product Manager & Glasses UX Designer |
Vivian Xin Shu
Product Manager & Designer |
Jayden DeCambre
iOS Engineer |
Dongchan Kim
Backend Engineer |
see you AWE2026