Emotion Detection Glasses
Emotion Detection Glasses
Collaborated with Katherine Lu, Evelyn Song, and Taarush Grover
An AR glasses application that recognize facial expressions to support autistic children’s social learning in a Stanford × Snap collaboration. Built the on-device emotion detection system in Lens Studio with custom scripts and shaped the UX through research with parents and educators.
IDEALIZATION PROCESS: Why Emotion Detection Glasses
Autism & Emotion Recognition
Notion-in-VR
| Pro | Con |
|---|---|
| Gesture recognition + voice to text is exciting | May reinvent an existing tool |
| Familiar use case (Notion, Miro) | Hard to stand out without a novel angle |
| High polish can impress | Gesture UX can be fragile |
Sensory Overload Reduction
| Pro | Con |
|---|---|
| Empathetic and novel | Hard to design one-size-fits-all |
| Strong real-world use cases | Subtle changes can be hard to demo |
| Pairs well with active input | Real-time filtering can be demanding |
Color Blindness Filter
| Pro | Con |
|---|---|
| Clear accessibility focus | Might not use full AR/VR interactivity |
| Easy to justify and story-tell | Hard to cover every color-blind type |
Social Coaching Glasses
| Pro | Con |
|---|---|
| Helps users with social anxiety or shyness | Social cues are nuanced and cultural |
| Uses camera + audio effectively | Emotion detection can misread cues |
| Flexible scope: start small, expand later | Balance comfort with feedback |
1. What is Autism?
- Autism Spectrum Disorder (ASD) is a neurodevelopmental condition.
- It affects communication, social interaction, sensory processing, and behavior.
- ASD is a spectrum — each person’s experience is different.
2. Why Focus on Children?
- 1 in 36 children is diagnosed with autism (CDC, 2023).
- Early intervention supports cognitive, emotional, and social development.
- Emotion recognition challenges begin early and can lead to:
- Social exclusion
- Communication breakdown
- Misunderstood behaviors
3. The Problem
- Children with autism may struggle to interpret facial expressions and body language.
- They can miss cues like sarcasm, frustration, or subtle social signals.
- They may experience emotional dysregulation without realizing it.
4. Why XR Can Help
- Children often learn better through play and visuals.
- XR glasses can augment reality with cues—colors, symbols, or words—to support emotion decoding.
- Builds empathy, confidence, and independence.
5. Our Vision
「To create a safe, intuitive, and personalized tool that empowers neurodivergent children to connect better with the world around them.」
THEORIES: The association between color and emotions
Iteration 1 Demo Video
User Testing Feedback:
- “Why would I associate emotion with colors?” → the link between color and emotion is not natural, need to be trained
- “I like the aura visual around the head. It makes me feel relaxed and centers my attention around the person’s face.”
- “I forget emotion-color association really quickly… Need some sort of reminder”
- “I like that it helps name feelings. Sometimes I know something’s off, but I can’t tell what exactly.”
- “If it interrupts while I’m thinking or trying to socialize, it’ll just make it worse.” / “Also, asking for permission for the person-at-interest to be detected by my camera is really awkward”
- “I really like the visual! The thing that doesn’t work for me is 1) the model doesn’t seem more accurate than my judgement 2) I feel uncomfortable being stared at by another person, knowing that they will decode what I think…”
Iteration 2 Demo Video
User Testing Feedback:
- Ethics – Precision can cross into privacy violation; the more accurate the tech, the more intrusive it feels.
- Nuance – Emotion isn’t just facial. Tone, gesture, and culture all shape how feelings are shown and read.
-
Utility – Constant detection might distract or make people feel watched.
- Biofeedback – Using signals like heart rate or brain activity can support self-awareness instead of surveillance.
Iteration 3 UI Design Choices
Atfer user testing, we arrived on this demo video:
Atfer user testing, we arrived on this demo video:
Final Demo Video
Key features added since Demo 3:
- Added animation at the beginning
- Differed sizes of the circles to identify the dominant emotion