AR based learning tool for autistic adults
AR based learning tool for autistic adults
AR based learning tool for autistic adults
Creating an accessible, inclusive learning experience for Luv Michael's Neurodivergent employees to learn and practice hygiene practices in their kitchens.
Creating an accessible, inclusive learning experience for Luv Michael's Neurodivergent employees to learn and practice hygiene practices in their kitchens.
Creating an accessible, inclusive learning experience for Luv Michael's Neurodivergent employees to learn and practice hygiene practices in their kitchens.
CLIENT
CLIENT
Luv Michael
(NYU School Project)
NASA, ARISA Labs
(NYU School Project)
NASA, ARISA Labs
(NYU School Project)
MY ROLE & TOOLS
MY ROLE & TOOLS
User Researcher & UX Designer
Figma, Snapchat's Lens Studio, Unity
TEAM & DURATION
TEAM & DURATION
3 designers under the guidance of Prof. Amy Hurst
6 weeks & Ongoing
Research current teaching methods, identify gaps, brainstorm new approaches, design a solution, and conduct usability testing.
Research current teaching methods, identify gaps, brainstorm new approaches, design a solution, and conduct usability testing.
Research current teaching methods, identify gaps, brainstorm new approaches, design a solution, and conduct usability testing.
The challenge
The challenge
The Safe Serv Handbook presents accessibility challenges for adults with autism (working at Luv Michael) at a preschool to kindergarten reading level. The current format fails to address their unique needs, impeding comprehension and use of essential safety information.
The Safe Serv Handbook presents accessibility challenges for adults with autism (working at Luv Michael) at a preschool to kindergarten reading level. The current format fails to address their unique needs, impeding comprehension and use of essential safety information.
The result
The result
Understanding the landscape
Understanding the landscape
Luv Michael has adult autistic employees who need an inclusive tool to learn about hygiene practices in the kitchen
Luv Michael has adult autistic employees who need an inclusive tool to learn about hygiene practices in the kitchen
Luv Michael has adult autistic employees who need an inclusive tool to learn about hygiene practices in the kitchen
Employees at Luv Michael had instructors training them in person, having an inclusive, interactive learning tool to reinforce/revise what they've learnt in these trainings about hygiene practices in the kitchen could be more effective.
Employees at Luv Michael had instructors training them in person, having an inclusive, interactive learning tool to reinforce/revise what they've learnt in these trainings about hygiene practices in the kitchen could be more effective.
To understand our users, how they learn, and work in the kitchen, we made a few client visits
To understand our users, how they learn, and work in the kitchen, we made a few client visits
To understand our users, how they learn, and work in the kitchen, we made a few client visits
Employees at Luv Michael had instructors training them in person, having an inclusive, interactive learning tool to reinforce/revise what they've learnt in these trainings about hygiene practices in the kitchen could be more effective.
Employees at Luv Michael had instructors training them in person, having an inclusive, interactive learning tool to reinforce/revise what they've learnt in these trainings about hygiene practices in the kitchen could be more effective.
Employees at Luv Michael had instructors training them in person, having an inclusive, interactive learning tool to reinforce/revise what they've learnt in these trainings about hygiene practices in the kitchen could be more effective.
So, what is needed? A learning tool for their curriculum keeping the below requirements in mind
So, what is needed? A learning tool for their curriculum keeping the below requirements in mind
So, what is needed? A learning tool for their curriculum keeping the below requirements in mind
Most employees understand kindergarten-elementary reading levels.
Some employees are non-verbal. Visuals, games, and hands-on activities are often helpful.
Visuals accompanied by audio is an effective approach.
Words of appreciation and motivation are essential for consistent learning.
They all possess phones and like to play games.
Customizable learning paths for varied learning abilities.
Most employees understand kindergarten-elementary reading levels.
Some employees are non-verbal. Visuals, games, and hands-on activities are often helpful.
Visuals accompanied by audio is an effective approach.
Words of appreciation and motivation are essential for consistent learning.
They all possess phones and like to play games.
Customizable learning paths for varied learning abilities.
Research and Brainstorming
Research and Brainstorming
Research and Brainstorming
Understanding Luv Michael's existing curriculum - ServSafe
Understanding Luv Michael's existing curriculum - ServSafe
We looked into Luv Michael's curriculum called ServSafe to understand their material and learn about how trainings are currently being carried out at Luv Michael.
We looked into Luv Michael's curriculum called ServSafe to understand their material and learn about how trainings are currently being carried out at Luv Michael.
Available tools/approaches in the market for neurodiverse population
Available tools/approaches in the market for neurodiverse population
We learned that:
Most available learning tools available in the market are for autistic children, and very few are available for autistic adults.
Many studies suggest that Augmented Reality and Autism work well together.
We learned that:
Most available learning tools available in the market are for autistic children, and very few are available for autistic adults.
Many studies suggest that Augmented Reality and Autism work well together.
Discussed our ideas with our Luv Michael POC
Discussed our ideas with our Luv Michael POC
Suggested using one of the following:
an interactive website game/video sessions
a tactile game
an Augmented Reality tool
Our client though the first two ideas were redundant since they already use those approaches. They wanted us to test if AR would work.
Suggested using one of the following:
an interactive website game/video sessions
a tactile game
an Augmented Reality tool
Our client though the first two ideas were redundant since they already use those approaches. They wanted us to test if AR would work.
User Testing with AR
User Testing with AR
User Testing with AR
To understand if AR is a potential solution, we wanted to see how Luv Michael employees interact with it
To understand if AR is a potential solution, we wanted to see how Luv Michael employees interact with it
To understand if AR is a potential solution, we wanted to see how Luv Michael employees interact with it
We did another client visit having some of the employees try using the below AR applications that we found online
We did another client visit having some of the employees try using the below AR applications that we found online
Chalk
Chalk
AR Basketball
AR Basketball
AR Apps
AR Apps
We took notes on how the employees interacted with their phones while using the application, and asked them what they liked about each application.
Most employees mentioned that they enjoy seeing movement on the screen during interactions (e.g., watching a basketball move). The instructor stressed the importance of considering employee preferences, as what they like often proves to be most effective for them.
The lead instructor at Luv Michael suggested that AR could be a promising approach and wants to try it out based on user testing.
We took notes on how the employees interacted with their phones while using the application, and asked them what they liked about each application.
Most employees mentioned that they enjoy seeing movement on the screen during interactions (e.g., watching a basketball move). The instructor stressed the importance of considering employee preferences, as what they like often proves to be most effective for them.
The lead instructor at Luv Michael suggested that AR could be a promising approach and wants to try it out based on user testing.
Designing the Experience
Designing the Experience
Designing the Experience
Hierarchical Task Analysis(HTA) is a crucial step when developing assistive technology
Hierarchical Task Analysis(HTA) is a crucial step when developing assistive technology
Hierarchical Task Analysis(HTA) is a crucial step when developing assistive technology
We did the below Hierarchical Task Analysis to understand the step-by-step process of how the user would use the product. This helped us anticipate different challenges they would face in each step and address the same.
We did the below Hierarchical Task Analysis to understand the step-by-step process of how the user would use the product. This helped us anticipate different challenges they would face in each step and address the same.
Building 'Hygiene in the kitchen' example scenarios and wireframes to make on Snapchat's Lens Studio
Building 'Hygiene in the kitchen' example scenarios and wireframes to make on Snapchat's Lens Studio
Building 'Hygiene in the kitchen' example scenarios and wireframes to make on Snapchat's Lens Studio
Crafting the UI Kit
Crafting the UI Kit
Crafting the UI Kit
Based on our research on how colors affect people with autism, we discovered the following:
A huge percentage of the population finds certain bright colors like reds and neon colors very intense.
Blues, greens, yellows, and pastel colors have a more positive effect on them.
The above research points led us to build the UI kit as follows.
Based on our research on how colors affect people with autism, we discovered the following:
A huge percentage of the population finds certain bright colors like reds and neon colors very intense.
Blues, greens, yellows, and pastel colors have a more positive effect on them.
The above research points led us to build the UI kit as follows.
First AR Prototype
First AR Prototype
First AR Prototype
We used Snapchat's Lens Studio to build our initial prototypes to be able to test out with the employees from Luv Michael to move to the next iteration.
Each scenario has readable text accompanied with slow audio and visuals while adhering to WCAG accessibility guidelines.
We used Snapchat's Lens Studio to build our initial prototypes to be able to test out with the employees from Luv Michael to move to the next iteration.
Each scenario has readable text accompanied with slow audio and visuals while adhering to WCAG accessibility guidelines.
Sample scenario #1 :
Which sink do we wash our hands in?
Sample scenario #1 :
Which sink do we wash our hands in?
Sample scenario #1 :
Which sink do we wash our hands in?
The first scenario tests the difference between the Hand washing sink and Kitchen sink at Luv Michael.
The first scenario tests the difference between the Hand washing sink and Kitchen sink at Luv Michael.
The first scenario tests the difference between the Hand washing sink and Kitchen sink at Luv Michael.
Uh-oh! Looks like there's something on the floor
Uh-oh! Looks like there's something on the floor
Uh-oh! Looks like there's something on the floor
After the First Scenario, the AR would ask the user to tilt their phone towards the floor and say, 'Uh-oh! Looks like there's something on the floor'. When they tilt the floor, they would see some virtual granola on the phone. The Avatar gives them an option to pick it up and the next scenario pops up after this action.
(We currently do not have this part prototyped.)
After the First Scenario, the AR would ask the user to tilt their phone towards the floor and say, 'Uh-oh! Looks like there's something on the floor'. When they tilt the floor, they would see some virtual granola on the phone. The Avatar gives them an option to pick it up and the next scenario pops up after this action.
(We currently do not have this part prototyped.)
After the First Scenario, the AR would ask the user to tilt their phone towards the floor and say, 'Uh-oh! Looks like there's something on the floor'. When they tilt the floor, they would see some virtual granola on the phone. The Avatar gives them an option to pick it up and the next scenario pops up after this action.
(We currently do not have this part prototyped.)
Sample Scenario #2:
After picking up something off the floor, what next?
Sample Scenario #2:
After picking up something off the floor, what next?
Sample Scenario #2:
After picking up something off the floor, what next?
The user is then prompted to select the right option after picking up something off the floor, which would be to wash their hands.
The user is then prompted to select the right option after picking up something off the floor, which would be to wash their hands.
The user is then prompted to select the right option after picking up something off the floor, which would be to wash their hands.
Next Steps
Next Steps
Next Steps
Going forward, we would like to:
1. Conduct usability testing with the current protoype with Luv Michael employees to note things that work and pain points.
2. Prepare detailed user flows of different types of scenarios.
3. Developing the AR experience on Unity using an iterative approach.
Going forward, we would like to:
1. Conduct usability testing with the current protoype with Luv Michael employees to note things that work and pain points.
2. Prepare detailed user flows of different types of scenarios.
3. Developing the AR experience on Unity using an iterative approach.