In-class review game
How can we create an engaging in-class review game?
An engineer, user researcher, product manager, and I had the
opportunity to research the classroom space and prototype how Khan
Academy might fit into a teacher's workflow! This project spanned
around 4 months from user research to concept testing.
This is my favorite project
from my time at KA! Sadly,
it never got realized
because of shifted company
priorities.
Research
Going into the user research study, our hypothesis was that teachers
want a homework replacement. We spoke to 13 mainstream math
teachers in the US about their day-to-day challenges. We sought out
teachers who didn't routinely use technology in their classrooms.
We learned a lot! These are some illustrations our user research lead
created to capture our learnings.
Teachers have to juggle so
many things at the same
time. It's definitely one of
the most underappreciated
professions out there 😭
Teachers usually teach at
grade level, but there are
students all over the
spectrum in their classes.
At a high level, we discovered that teachers didn't find assigning
homework challenging, but that
keeping student engaged and knowing
where they are is tough.
We also created some design principles to guide our thinking for the
rest of the project:
Brainstorming
We took over a corner of the
office and called it the
foam board fort.
We created How Might We (HMW) questions and invited the rest of the
company to generate ideas with us! Some of these questions included:
HMW make students feel joyful filling their own gaps for classrooms
with no access to tech?
HMW make a fun diagnostic for teachers who know tech is a classroom
management issues?
HMW make diagnostics feel useful on a routine basis for teachers on
autopilot?
At the same time, we looked at existing classroom products that
teachers loved! We realized that in-class review games like Kahoot!,
QuizletLive, and Socrative had high rates of growth.
Through ideation & competitive analysis, we narrowed our question to:
How might we create an engaging in-class math review game? For the
following couple months, we used this question to guide our
prototoype.
Game loop
First, we sketched out the core game loop.
Storyboarding
Then, we drew out the general flow of the game and how it would fit
into the classroom.
A. Ms. Frizzle is planning
the lesson for tomorrow and
sees that it's almost time
for the Unit 2 test. She
decides to use Khan Academy
for the review session.
B. Ms. Frizzle pulls up up a
question on the screen using
Khan Academy. Each student
tries to solve the problem
on their own device or in
groups.
C. Ms. Frizzle sees that
almost half the class
answered incorrectly and
decides to explain the
question at the board.
Game jam: prototyping
I can't show the footage of students in my public portfolio, so these are
all low-fi gifs! If you want to see the final prototype live in action,
please visit
khanacademy.org/r/classroom-teacher
and select the
"Demo" game.
Once we had the basic game mechanics outlined, we started to
prototoype the game. Each week, we did the following:
🤔 Hypothesis
💻 Prototype
Playtest in a classroom
Debrief with the teacher
🔁 Iterate
✨ Round 1.
The suspense of waiting for the right answer
will be engaging for students.
We created the prototype in Framer. The Desktop view on the left is
what the teacher projects on the screen and the Mobile view is the
student screen.
In these stills we used a
mobile screen for students
to differentiate from the
teacher view. In most
classrooms, students use
Chromebooks.
We wanted the sign in to be as easy as possible, so instead of having
teachers roster their students, we wanted students to join the game
through a simple class code.
In our original game loop, we assumed that teachers would
dramatically "reveal" the answer to the class. During the first week, we
learned that this hypothesis was incorrect. The suspense caused
students to be distracted because they didn't know what to do after
they answered a question.
After playtesting, we also realized that this design was hard for
teachers to interact with during class. It was hard to tell which
students had answered the question right or wrong.
✨Round 2.
Teachers want to engage with the data live
during class.
In the 2nd iteration, we pivoted the design to have a large sidebar with
the students. When a student got a question correct, a large green
check appeared next to their name. We drew the focus away from the
avatars because those were the least important part and mostly a fun
addition for the students.
We realized it was too hard for teachers to engage with all the
information immediately, so for the following iteration we wanted to
explore what an after-class teacher report might look like.
✨Round 3.
Teachers want to diagnose patterns in student
learning after class.
The results below are from an internal testing session, so the data isn't
very accurate. We wanted to be transparent with teachers and show
them all the raw data, so that they could make their own assumptions.
Yellow in correctness means that a student got the question correct
eventually.
Teachers loved being able to diagnose patterns in student
performance, so we decided to follow up the next week with a student-
facing post game experience.
✨Round 4.
Students want to study using their results from
in-class practice.
We wanted students to keep engaging with Khan Academy content
after the game by watching videos or reading articles to fill their gaps
in learning.
✨Round 5.
Students will be more motivated to do well
with higher stakes.
We collaborated with a game designer this week to discuss how we
could engage students further. Previously, the game was just a series of
exercises. There was no motivation for students to do well other than
the fact that their teacher would see their answers!
When a student joined the game, they would get placed on either team
Green or team Blue. Each student would get 10 points for getting a
question correct and 1 point for trying. All students would max out at
10 points per question.
When we tested this in the classroom, students would cheer for their
teams and try to help each other get the right answer!
If you want to see the final prototype live in action, please visit
khanacademy.org/r/classroom-teacher
and select the "Demo" game.
This is a super low quality gif of what it looked like in the classroom:
Future explorations
While we were prototyping ideas in the classroom, I also drew more
"far out" explorations to show how the product could look if we
expanded upon the idea even further. All these explorations focus on
how the class or team will collectively reach a goal. We didn't want the
focus to be on an individual student.
During projects, I like to
have a separate stream of
work to explore more far out
ideas! I think it's easier
come up with good solutions
by dreaming up the future
before simplifying it.
In this iteration, each team is a represented by a plant. Every time a
student answers a question, they get to contribute to the plant's
growth! I replaced the emojis with a sun and cloud because both
sunshine and rain are needed to grow a plant.
I'm pretty sure I proposed a
garden metaphor for every
project I worked on at Khan
Academy 😬
Khan Academy has a partnership with Pixar, so I wanted to explore
how we might be able to bring that world into the product. In this
example, every time a student gets a question right, they get the
opportunity to throw a balloon onto the UP house. The goal for the
class could be to get enough balloons to make the UP house fly away.
In this exploration, students would be fighting a Monster of the Day.
Every day, the monster would start out with X amount of HP and the
class would have to defeat the monster by answering questions.
Teachers love to have students reason through their answers! In the
following exploration, we would surface two popular answers and
select two students to "debate" the solution. Other students can send
hearts to the two debaters based upon who they think is correct!
What happened?
Although we received lots of teacher validation during the project, the
leadership team decided to hold off on pursuing this idea until we had
a more solid foundation for the teacher product.
Previous
Teaching visual design
Fashion averages
Next
Home
About
Design
Teaching
Illustration
Experiments