Project Overview

Challenge

Since the inception of the Smarter Balanced Summative Assessments from the California Assessment of Students Performance or Progress (CAASPP), more than half of LVJUSD's students have scored below standard (grade level) in math. Results also show a steady decline in student's scores as they age. Survey results show that many students are receiving little to no support at home. Families are struggling to have positive experiences with math and have categorized math as a high anxiety subject.

Solutions

Motivate students and get them excited by exposing them to math in the real world
Connect families with local businesses and services that support the overall wellbeing of the child such as dental offices, health clinics, libraries, etc.

Outcome

Released to over 10k students and families every year
Generates $100k+ in revenue for participating businesses.
Has received sponsorship from top companies such as Carl's Jr and Jamba Juice
Increase in satisfaction of teachers polled regarding family involvement with math

Role

Creator, Product Designer

Task

Create a math program for the school district that would help students improve their math abilities.

Tools

Surveys, Stacked Bar Chart, Empathy Map, Personas, Point of View (POV), How Might We (HMW), Venn Diagram, Project Plan, Affinity Map

My Process

Empathize

>

Define

>

Ideate

>

Prototype

>

Test

Empathize

Research

Research Objectives

I used both quantitative and qualitative research methods to gain a detailed understanding of the student's in the pilot classes' math abilities.

Measure students' skills across state math standards
Identify and evaluate strengths and weaknesses in math
Understand student and family math experiences

Quantitative Research

Key insights that I discovered about student's performance and background from previous State Assessment Results and Demographic data:

Trends & Insights

As the student's aged their scores declined. From 3rd to 4th grade, there was a 20% decrease in students being classified as working at grade level.
6.56% of students were classified as above standard for mathematical procedures and rules. There is a lot of opportunity for improvement in this grade level.
9.84% of students could apply and show their problem solving skills according to California standards for fifth grade. Students were not understanding how to solve math problems.

Demographics

90% of the students were first generation Americans
82.4% were Hispanic
91.1% of students were from low-income families
70.5% were classified as English Language Learners; Spanish was the primary language spoken in most households
58% of the students identified as Male and 42% identified as Female

Qualitative Research

To build upon my quantitative insights, I created a survey for students and their families and set up one-on-one interviews.

Survey Results

My survey's primary focus was on the families' perception of the child's comfort with math as well as their own comfort level supporting their child in math. I found that families had limited understanding of how their kids were doing in math and they were not comfortable providing math guidance. At the most basic level, many parents were not talking to their children about math.

Interviews & Empathy Map

My one-on-one interviews gave me a deeper understanding of the experiences students and families have with math. Each interview lasted around 15-20 minutes. During the interviews, I went over their survey results and asked open-ended questions to clarify responses.

To synthesize all the information that I had gathered from the user interviews, I took notes and created an empathy map. By identifying common patterns across my findings, I was able to uncover key insights that informed the design of the math passport.

Defensive Mechanism

Parental Involvement

Goal Setting

Findings

Most student's struggles caused them to question the relevancy of math outside of school.

Most students had little to no discussions about math with their families.

Students did not take ownership over their own goals in math.

Insight

Student's defense mechanisms allowed many to remain complacent in their current position.

With limited parental involvement, students lacked advocacy and accountability for math progress.

Students lacked understanding of what they are working towards or why they are doing it.

Needs

Students needed to be shown that math is relevant outside of school.

Students needed to have more opportunities to discuss math with their families.

Students needed to take ownership over their own learning.

User Persona

Using what I learned from the previously mentioned research, I created a user persona to ensure my design decisions were user-centric. Meet Alejandro!

Define & Ideate

Defining the Problems

To help drive my brainstorming process, I built upon the insights gleaned from both the quantitative and qualitative research by identifying the key needs from my insights, possible Point of Views (POV) using my target persona and How Might We (HMW) questions that addresses each need.

Insights

Needs

POV

HMW

Student's defense mechanisms allowed many to remain complacent in their current position.

Students needed to be shown that math is relevant outside of school.

Alejandro loves pizza and is interested in learning how to pay for his pizza with his friends.

How might we connect Alejandro with the local pizza shop to help him understand how math is used in their business?

With limited parental involvement, students lacked advocacy and accountability for math progress.

Students needed to have more opportunities to discuss math with their families.

Alejandro and his family only talked about math after report cards were released.

How might we help Alejandro and his family find fun and stress-free ways to talk about math?

Without ownership over their own math goals, students lacked understanding of what they are working towards or why they are doing it.

Students needed to take ownership over their own learning.

Alejandro finds it hard to ignore distractions when he doesn't understand what he is suppose to do or why he is doing work.

How might we discover what would motivate Alejandro to engage in his own learning of math?

Ideating Solutions

I brainstormed multiple solutions to each How Might We question and combined them to create the Math Passport program.

The Math Passport is designed to guide students through an immersive, augmented reality adventure that will take them and their families all over the Livermore community, while challenging them with standards-aligned math questions. Their devices use GPS to direct them to a variety of locations, and ask them to solve a math question once they arrive at each stop. The correct answer is the “Math Password” that will earn the student points.

Project Goals

In order to come up with the projects' goals, I assessed the interest of all participating parties and focused on where they aligned.

Primary goal ended up being "Guide students and families through an augmented reality adventure that improves math abilities."

Prototype & Testing

Pilot Approach

The piloting of the program was divided into three phases:

Phase 1: Onboarding students, businesses and financial backers
Phase 2: Usability testing and feedback
Phase 3: Final iteration and handoff to district administrators

Phase 1: Onboarding students, businesses and financial backers

I used my understanding of Alejandro to choose 15 places that would be the most appealing and beneficial to him and his family. The selected sites ranged from places offering free health services to a town favorite donut shop. After creating my list, I contacted the businesses to confirm participation.
‍
I then presented the list of sites to the students and held a feedback session. During the session, I was reminded of a variety of constraints and concerns. For example, family time constraints was a common concern so I accounted for this by selecting places that students would be able to visit outside of parent work schedules. I then narrowed the list down to 10 places.

Below: Initial list of sites, expanded list after student feedback, final list.

Phase 2: Usability testing and feedback

I observed student and family interactions at various sites throughout the pilot. Upon completion of the pilot, students were asked to complete a survey and I held one-on-one interviews. My goal for the feedback sessions was to understand what they were doing, thinking, and feeling while participating in the pilot to see if my overall goals for the project could be met.

Below is the timeline that I used for the usability test.

Pilot Statistics

Participants: 34
Grade: 5th
Average Time: 6 Days
Completion Rates: 94.1%

Overall, the testing showed positive results, but there were distinct areas in which students and their families did face difficulties.

Affinity Map

To synthesize my findings from the testing observations and interviews, I took all my notes and created an affinity map. This helped me to better digest the different patterns observed and pin point where revisions need to be prioritized to improve the usability of the design.

Based on the prominent patterns observed related to the pain points, I was able to uncover insights which helped lead to specific design recommendations.

Pain Points

Insights

Recommendations

Difficulty with solving math problems

61% expressed having trouble solving some of the math paroblems

I would provide hints to each problem that students would find at the sites

Limited prize options

Many believed that the variety of prizes should not be limited to certain individuals (ie..those with dietary restrictions)

Everyone would be eligible for all the prizes

Pilot Length

55% felt rushed by the pilot's time limit

I would provide more time

Taking what I learned from my affinity map, I began to make revisions to my design based on the recommendations identified.

1. Provide a hint to assist in solving the problem.

During the testing and subsequent interview process, I found that many of the participants struggled with the math problems. A hint regarding how to solve the problem would have been useful.

Before & After

2. Provide more of a variety of prizes in each place.

Originally, I had spoken with the food sites about providing "other" options for a few of the students who had certain dietary restrictions. For example, if a student had a dairy allergy, they could choose a different type of donut than the donut-holes which had dairy. During my interviews, it was suggested that this be a more widespread option.

3. Extend given time for program completion.

Students and their families were given 10 days to complete the program. I was concerned that more time would put businesses in an uncomfortable position and did not want to request that of them. However, 10 days would not be enough time for thousands of families across the district to complete the program and would overwhelm the sites and businesses.

Phase 3: Final iteration and handoff to district administrators

After seeing the success of the usability testing, I decided to scale the program. I performed qualitative and quantitate research methods such as analyzing data from 11 elementary schools including demographics, testing results, etc. I also created a family survey to create a couple of target personas for the district as a whole.

Once I created my target personas, I identified the best businesses to participate in the district wide math passport program. I then presented my plans to the superintendent for approval. The project was accepted by the district, received financial sponsorship from the Livermore Valley Education Association and Livermore Chamber of Commerce. It was handed off to the District's Utilizing New and Innovative Technologies in Education (Unite) Team to be developed into an app and released to all 5k+ elementary students during the summer of 2021.

Below is one of the target personas that I created for the district.

Reflection & Next Steps

I really enjoyed working on this project. It was a very unique experience for me because I had the autonomy to design something from scratch that met the needs of my students. I learned a lot from working with the donors, businesses, district administrators, staff, and students.

This project required a lot of analytical research and pushed me to be resourceful in finding what I needed.

Things that I would do differently now:

1. More Inclusive

Make the project more inclusive for the hundreds of students with special needs.

2. Scale

I would like to have the program include upper grade students.

< DYM: Life Coaching & Therapy Your Teacher Village >