AR-based learning tool for Luv Michael's autistic employees
AR-based learning tool for Luv Michael's autistic employees
AR-based learning tool for Luv Michael's autistic employees
Creating an accessible, inclusive learning experience for Luv Michael's Neurodivergent employees to learn and practice hygiene practices in their kitchens.
Creating an accessible, inclusive learning experience for Luv Michael's Neurodivergent employees to learn and practice hygiene practices in their kitchens.
CLIENT
CLIENT
Luv Michael
NYU school project. (Assistive Technology Course)
Role
Role
UX Designer
AR Developer
Team & Duration
Team & Duration
3 UX Designers,
2 AR Developers.
Six Weeks & ongoing
Tools
Tools
Figma
Snapchat's Lens Studio
Unity
The Problem -
The Problem -
The Problem -
From training to practice - the gap remained. Autistic employees need a better way to reinforce personal hygiene training in the kitchen.
From training to practice - the gap remained. Autistic employees need a better way to reinforce personal hygiene training in the kitchen.
the solution -
the solution -
the solution -
An AR-based learning tool to practice trainings in 10 min interactive exercises.
An AR-based learning tool to practice trainings in 10 min interactive exercises.
impact
impact
impact
And solving this mattered because:
And solving this mattered because:
And solving this mattered because:
60% increase in user engagement - employees were engaged in learning about safety practices.
Reduced cost and time to reinforce safety trainings by 20%.
Further usability testing is planned to measure information retention.
A project involving an interactive accessible map designed to guide "citizen scientists" to analyze eclipse data, ensuring that every observer, regardless of their physical abilities, could partake in this experience, particularly members of the Blind and Low-Vision community.
action
action
action
What led to the solution?
What led to the solution?
Field research - observing employees in the kitchen
Competitive analysis
Usability testing
UI Kit design
Prototyping
AR Tool - design & development
Field research - observing employees in the kitchen
Competitive analysis
Usability testing
UI Kit design
Prototyping
AR Tool - design & development
Field research - observing employees in the kitchen
Competitive analysis
Usability testing
UI Kit design
Prototyping
AR Tool - design & development
Now, let's take a closer look at the problem
Now, let's take a closer look at the problem
Now, let's take a closer look at the problem
CONTEXT
CONTEXT
CONTEXT
Luv Michael’s autistic employees needed a more accessible way to learn safe kitchen practices.
Luv Michael’s autistic employees needed a more accessible way to learn safe kitchen practices.
Luv Michael’s autistic employees needed a more accessible way to learn safe kitchen practices.
What is Luv Michael? - A granola making non-profit aimed at providing employment for Autistic adults.
What is Luv Michael? - A granola making non-profit aimed at providing employment for Autistic adults.
Currently, they receive training through instructor-led lessons and physical games. However, given the diverse learning abilities of the employees, Luv Michael sought a more inclusive tool to better support their needs.
Currently, they receive training through instructor-led lessons and physical games. However, given the diverse learning abilities of the employees, Luv Michael sought a more inclusive tool to better support their needs.
The Challenge
How might we make Luv Michael’s personal hygiene lessons more accessible for its autistic employees?
How might we make Luv Michael’s personal hygiene lessons more accessible for its autistic employees?
How might we make Luv Michael’s personal hygiene lessons more accessible for its autistic employees?
RESEARCH
RESEARCH
RESEARCH
Field Research: Stepping into the kitchen and observing challenges first hand
Field Research: Stepping into the kitchen and observing challenges first hand
Instructors at Luv Michael use a curriculum called ServSafe to train their employees on safe kitchen practices.
Instructors at Luv Michael use a curriculum called ServSafe to train their employees on safe kitchen practices.
Lessons are conducted by the instructor using a whiteboard, where she writes and draws as she speaks. To reinforce these lessons, employees engage in physical card sorting activities.
Lessons are conducted by the instructor using a whiteboard, where she writes and draws as she speaks. To reinforce these lessons, employees engage in physical card sorting activities.
Why is the current learning system not effective?
Why is the current learning system not effective?
Low information retention.
Low information retention.
Group learning was tricky- varied learning abilities.
Group learning was tricky- varied learning abilities.
Employees had difficulty applying lessons in kitchen.
Employees had difficulty applying lessons in kitchen.
What is the impact?
Cost and time in lesson repetition.
Cost and time in lesson repetition.
Hygiene in the kitchen is compromised.
Hygiene in the kitchen is compromised.
Company reputation might be impacted.
Company reputation might be impacted.
What might help?
What might help?
What might help?
Quick, simple exercises to reinforce learning.
A degree of personalized learning.
A degree of personalized learning.
An immersive, practical experience to connect training with real-life tasks.
An immersive, practical experience to connect training with real-life tasks.
Secondary Research: Exploring available methods for neurodivergent learning
Secondary Research: Exploring available methods for neurodivergent learning
Understanding Luv Michael's curriculum - I reviewed Luv Michael's ServSafe curriculum to see their material, understand the content that is being taught and how.
Understanding Luv Michael's curriculum - I reviewed Luv Michael's ServSafe curriculum to see their material, understand the content that is being taught and how.
Studies on Autism and Learning suggest that - augmented reality is quite effective for learning in autistic individuals.
Understanding Luv Michael's curriculum - I reviewed Luv Michael's ServSafe curriculum to see their material, understand the content that is being taught and how.
Understanding Luv Michael's curriculum - I reviewed Luv Michael's ServSafe curriculum to see their material, understand the content that is being taught and how.
Studies on Autism and Learning suggest that - augmented reality is quite effective for learning in autistic individuals.
I conducted a brainstorming for new learning methods session with my team & Luv Michael
I conducted a brainstorming for new learning methods session with my team & Luv Michael
Below are some of the options we explored and the reasons why they work/don't.
Gamification - Web or Tactile Game?
Why might they work?
Why might they work?
Gamification - can make it engaging.
User can control the pace for the web game.
Can use audio, visual, and text catering to different learning styles.
Moving and touching physical cards can be an effective learning technique.
Gamification - can make it engaging.
User can control the pace for the web game.
Can use audio, visual, and text catering to different learning styles.
Moving and touching physical cards can be an effective learning technique.
Gamification - can make it engaging.
User can control the pace for the web game.
Can use audio, visual, and text catering to different learning styles.
Moving and touching physical cards can be an effective learning technique.
Why not?
Why not?
The current material used to teach at Luv Michael is similar to the web game.
Want something that does not require assistance from another person.
The current material used to teach at Luv Michael is similar to the web game.
Want something that does not require assistance from another person.
An Augmented Reality based tool?
Why might it work?
Why might it work?
Studies show that AR works well for people with Autism.
Training in context helped employees connect learning to their job.
Walking around the kitchen might increase connectivity of lessons with kitchen experiences.
Our Luv Michael contact saw potential and encouraged further AR exploration.
Studies show that AR works well for people with Autism.
Training in context helped employees connect learning to their job.
Walking around the kitchen might increase connectivity of lessons with kitchen experiences.
Our Luv Michael contact saw potential and encouraged further AR exploration.
Why might it work?
Integrating the work environment into training helps employees relate learning to their job.
Our Luv Michael contact, found this approach promising and wanted us to explore AR further.
Luv Michael wanted us to test out an AR-based approach.
Luv Michael wanted us to test out an AR-based approach.
USER TESTING WITH AR
USER TESTING WITH AR
USER TESTING WITH AR
Observing ten Luv Michael employees' interactions with AR for learning.
Observing ten Luv Michael employees' interactions with AR for learning.
My team and I conducted fly on the wall observations noting how employees interacted with their phones while using the AR applications. Below are the details:
7 employees (chosen by Luv Michael)
15 min each
3 applications
My team and I conducted fly on the wall observations noting how employees interacted with their phones while using the AR applications. Below are the details:
7 employees (chosen by Luv Michael)
15 min each
3 applications
Chalk
Chalk
Chalk was selected to test how employees interacted with:
a free-flowing
minimal-constraint AR experience.
PhD students struggle to find academic resources, events, and support in one place.
AR Basketball
AR Basketball
AR Basketball was selected to observe how employees responded to a simple, structured AR experience.
A lot of times, students weren't even aware of events happening or services available to them.
AR Paddle Ball
AR Paddle Ball
AR Paddle Ball was selected to observe how users engaged with a more flexible, less structured experience.
AR Paddle Ball was selected to observe how users engaged with a more flexible, less structured experience.
This user testing session helped us understand specific practices that work better than others and that AR was worth exploring.
This user testing session helped us understand specific practices that work better than others and that AR was worth exploring.
DESIGN - ITERATION #1
DESIGN - ITERATION #1
DESIGN - ITERATION #1
I designed user flows, scenarios, and wireframes for the AR feature.
I designed user flows, scenarios, and wireframes for the AR feature.
With the project's initial 4-week timeline, Lens Studio was chosen as it offered the shortest learning curve for building the AR feature.
With the project's initial 4-week timeline, Lens Studio was chosen as it offered the shortest learning curve for building the AR feature.
User Flow
User Flow
User Flow
User Flow
Low-fi wireframes
Low-fi wireframes
Low-fi wireframes
Low-fi wireframes
To explore how basic interactions worked, we designed a basic AR split-screen experience
To explore how basic interactions worked, we designed a basic AR split-screen experience
We used Snapchat's Lens Studio to build our initial prototype to conduct a round of usability testing with Luv Michael's employees and move to the next iteration.
Readable text.
Text accompanied by slow speed audio and visuals. - to cater to different learning needs.
Adhering to WCAG guidelines.
We used Snapchat's Lens Studio to build our initial prototype to conduct a round of usability testing with Luv Michael's employees and move to the next iteration.
Readable text.
Text accompanied by slow speed audio and visuals. - to cater to different learning needs.
Adhering to WCAG guidelines.
Sample Scenario #1: Which sink do we wash our hands in?
Sample Scenario #1: Which sink do we wash our hands in?
Uh-oh! Looks like there's something on the floor!
Uh-oh! Looks like there's something on the floor!
After the first scenario, the AR tool prompts users to tilt their phone towards the floor, showing virtual granola on the screen.
After the first scenario, the AR tool prompts users to tilt their phone towards the floor, showing virtual granola on the screen.
Sample Scenario #2: What do we do after picking something off the floor?
Sample Scenario #2: What do we do after picking something off the floor?
Testing the first prototype with 10 employees helped us understand what not to do
Testing the first prototype with 10 employees helped us understand what not to do
SURPRISE ITERATION #2 - PIVOT!
SURPRISE ITERATION #2 - PIVOT!
SURPRISE ITERATION #2 - PIVOT!
Luv Michael wanted us back, and we were excited to keep going!
Luv Michael wanted us back, and we were excited to keep going!
Luv Michael wanted us to continue working on the prototype beyond the initial project timeline and we were thrilled!
A lot has changed and now I understood that it was important to pivot.
Luv Michael wanted us to continue working on the prototype beyond the initial project timeline and we were thrilled!
A lot has changed and now I understood that it was important to pivot.
First Iteration
Second Iteration
Snapchat's Lens Studio: Limited scope that doesn't suit new iteration.
Snapchat's Lens Studio: Limited scope that doesn't suit new iteration.
Unity AR: Higher technical feasibility
Unity AR: Higher technical feasibility
Camera open throughout: distracting
Camera open throughout: distracting
Camera open only while setting the scenario: clear indication
Camera open only while setting the scenario: clear indication
Split screen approach: Confusing
Split screen approach: Confusing
One screen with 2 options side-by-side: clear and simple decision making
One screen with 2 options side-by-side: clear and simple decision making
AR focus throughout: Doesn't help in effective scenario building
AR focus throughout: Doesn't help in effective scenario building
AR Focus only at essential aspects: for effective scenario building with emphasis on understanding the question at hand.
AR Focus only at essential aspects: for effective scenario building with emphasis on understanding the question at hand.
The new requirements demanded new user flows, wireframes, and a complete UI Kit.
The new requirements demanded new user flows, wireframes, and a complete UI Kit.
The following user flow diagram was updated for the same scenario from the first iteration, reflecting changes based on user feedback and language adjustments from Luv Michael’s instructors.
The following user flow diagram was updated for the same scenario from the first iteration, reflecting changes based on user feedback and language adjustments from Luv Michael’s instructors.
The following user flow diagram was updated for the same scenario from the first iteration, reflecting changes based on user feedback and language adjustments from Luv Michael’s instructors.
Revised Low-fi Wireframes
Revised Low-fi Wireframes
Revised Low-fi Wireframes
Revised Low-fi Wireframes
Revised UI Kit
Revised UI Kit
Revised UI Kit
Revised UI Kit
FINAL DESIGN
FINAL DESIGN
FINAL DESIGN
High-fidelity screens were designed in Figma and transformed them into a Unity AR app.
High-fidelity screens were designed in Figma and transformed them into a Unity AR app.
As shown below, decisions like the color palette, minimal AR use, and the two-option approach were all refined based on user testing and insights from earlier iterations.
As shown below, decisions like the color palette, minimal AR use, and the two-option approach were all refined based on user testing and insights from earlier iterations.
SHOWCASING OUR WORK
SHOWCASING OUR WORK
SHOWCASING OUR WORK
Our work was showcased at Tandon's Research Excellence Exhibit 2024, where we got valuable feedback.
Our work was showcased at Tandon's Research Excellence Exhibit 2024, where we got valuable feedback.
At Tandon's Research Excellence Exhibit, we got a chance to present our research and prototype to numerous attendees and judges. The feedback that the participants gave us based on their use of the prototype and the research was helpful in thinking about future considerations.
At Tandon's Research Excellence Exhibit, we got a chance to present our research and prototype to numerous attendees and judges. The feedback that the participants gave us based on their use of the prototype and the research was helpful in thinking about future considerations.
REFLECTIONS
REFLECTIONS
REFLECTIONS
Turning feedback chaos into clarity
Turning feedback chaos into clarity
Turning feedback chaos into clarity
Working with Luv Michael was a rewarding experience, allowing me to lead the end-to-end design and development process.
Collaborating with a neurodiverse audience honed my ability to adapt to evolving requirements while prioritizing accessibility and inclusivity.
A key takeaway was the importance of being flexible and ready to pivot as conditions change.
I learnt that sometimes it is perfectly alright to spend time learning what not to make.
Working with Luv Michael was a rewarding experience, allowing me to lead the end-to-end design and development process.
Collaborating with a neurodiverse audience honed my ability to adapt to evolving requirements while prioritizing accessibility and inclusivity.
A key takeaway was the importance of being flexible and ready to pivot as conditions change.
I learnt that sometimes it is perfectly alright to spend time learning what not to make.
What would I do next?
What would I do next?
What would I do next?
Conduct usability testing with the latest prototype on Luv Michael's iPad to identify pain points and effective learning approaches for future iterations.
Capture 3D scans of kitchen equipment to integrate into the app, enhancing relevance for employees.
Expand the app to cover additional chapters of the book for a more comprehensive learning experience.
Conduct usability testing with the latest prototype on Luv Michael's iPad to identify pain points and effective learning approaches for future iterations.
Capture 3D scans of kitchen equipment to integrate into the app, enhancing relevance for employees.
Expand the app to cover additional chapters of the book for a more comprehensive learning experience.