Students Using Khan Academy on iPads

Data-Driven Math Instruction: A Gradual Approach

In a previous post, I described my first steps toward creating a gamified math class using Khan Academy. I didn’t set out with gamification as a goal. At first, I was mostly interested in Data-Driven Instruction. Introducing Khan Academy helped me to assign different work to different students and assess their progress, while reducing the time I spent grading and planning. At first, I used Khan Academy as a tool for homework and as a resource for teacher-centered instruction. Once I had done some exploring, I realized the potential of Khan Academy to completely transform the dynamics of the classroom.

There are elements of gamification built right into Khan: students receive points for achievement and they have some flexibility in their learning path. I eventually developed a gamified and collaborative learning model around the platform. I modified classroom routines and expectations – even the content students were expected to cover. It required letting go of a certain amount of control I expected to have as a teacher. In the end, however, my students ended up learning a lot more math and enjoying the journey as well.

The Power of Points

After rolling out the Khan Academy homework program in January, I soon began gathering data to better understand my students. For the first time, I had an objective measure of the amount of time students spent on their work. In addition, Khan had built-in gamified elements that were surprisingly effective – students received “energy points” for watching videos and answering questions; in addition, badges recognized larger accomplishments, and status indicators measured their progress through grade levels.

Graph of Student Activity from Khan Academy
Graph of Student Activity from Khan Academy

I initially set a goal of 20 minutes per day, spread across any 5 days each week. I thought that making homework time flexible would give students more ownership over their progress. I recognized that students have active lives outside of school – rather than e-mailing back and forth with parents asking their child be excused because their soccer game ran late the night before, students could simply take that night off and complete their minutes another night.

Once Khan Academy provided me data on “time spent,” it became immediately clear that I had been making several errors in assessing effort. In some cases, I had accurately estimated how much time students were spending on their work. In others, I had credited bright students with more time than they actually spent. I had also failed to appreciate the effort of some struggling students. While measuring the amount of time students were spending on homework was helpful, it was only the first step – I had to make sure that those who were putting in effort were able to maintain it, and those who were not would improve.

Fortunately, I found that just such a mechanism was built right in to Khan Academy. The first element of gamification that caught on in my class were Khan’s ‘energy points,’ which students earned by watching videos or answering questions. At the time, I was completing activities on Khan Academy as a student so I would know what my students were encountering. Since I could go through problem sets quickly, I earned a lot of points in a short period of time, and when I projected activities on the board, students would point out how many points I had earned. Soon, a little competition developed where a number of students tried to beat my energy point score. One student in particular (if you’re reading this, you know who you are!) was so determined that he spent many times the required minutes, and soon I was unable to keep up with him.

Before we began, I thought some of these were a little silly, and I doubted if students would really care about earning points and badges on a math website, but I was surprised when I saw the excitement it added to the class. Since every student is motivated differently, it was helpful to have a number of ways to encourage different students. Khan also began e-mailing me a weekly ‘leaderboard’ that listed students in order of the number of minutes they had completed that week. Every Monday, I would show the top students on the leaderboard. The students who had not made the leaderboard weren’t called out for it, but most of them really wanted to make it up there for the next week. Sometimes I would use “promotions” such as offering a trip to the park if everyone made the leaderboard for the week.

The First Measurable Result: Effort

Soon, homework completion in my classes had improved significantly, which also helped increase engagement in lessons and activities. As time went on, I tailored the specific rules in response to what I heard or observed. For one, some parents complained that 20 minutes 5 nights a week (100/week) was too much, so my principal and I decided 15 per night (75/week) would be sufficient.

I also had to adjust my grading policies to account for the quality and not just the quantity of time spent. Since I entered the homework grades each week, some students would simply wait until Sunday night and do all 75 minutes in one night, which I knew was a much less effective learning strategy than spreading time out over the week. Students could earn energy points for watching videos, and some would just set up a video to play on their computer while they did other things.

One inventive student even realized he could play movies in several windows at the same time and complete a week’s worth of homework in ten minutes. If he hadn’t chosen to use the same video, I might not have even noticed! I then added a rule that students would only get credit for the time from a video if they completed the problem set associated with it. It could be a bit tedious to play detective on what activities students completed, but I really only needed to look deeper if I thought something didn’t seem right – the goal was creating a culture in which students come to appreciate why they are doing the work, instead of forcing them to do work.

Get the support you need to bring personalized learning to your school:

Measuring and Supporting Achievement

I had initially decided to focus solely on effort at the beginning of my implementation, and I believe that was the right decision. It gave my students and I time to adjust to the platform. Also, If I had tried to measure student growth and proficiency all at the same time, I would not have been able to keep track of everything. Time on platform was an essential foundation which allowed me to more effectively begin measuring other aspects of performance.

I soon noticed that some students were skipping the recommendations I had made and exploring other topics. In some cases, I thought it was great that an advanced student was trying out Trigonometry; when a student struggled with dividing whole numbers, though, and was instead spending time on Calculus (unsuccessfully) or “Counting to 5” (even successfully), we would have to have a talk. I sometimes had to award less than full credit to students who completed their minutes but seemed to be ‘gaming the system.’

Since I wanted to encourage exploration, though, I told them that they needed to spend at least half of their time on my recommendations, and they could spend the other half on any activities they chose. As time went on, I encountered a number of similar situations and had to make value judgments about when I would follow their lead and when to insist they follow mine.

Recommendations page at khan academy
Recommendations page

While energy points and minutes of homework were instrumental in getting the blended learning program started, at some point, I needed to involve other metrics of student progress. As much as I liked the fact that students could work at their own level, it was becoming clear that some students had not stepped up to take ownership as I stepped back my “management” of their learning. Still, I didn’t want to cater to the lowest common denominator and revert to a system where everyone does the same work, so I had to look for alternatives.

Grade level progress for a full class
Grade level progress for a full class

I began setting guidelines about the minimum grade level progress I expected. Some students had already mastered all of the 4th and 5th grade level course work on the platform, while others had only completed about 30% of 4th grade. I set class goals, such as ‘everyone completes at least 50% of 4th grade’ by a certain date, gradually moving the bar forward. That way, advanced students could move ahead and other students could work on remedial material but still know when they needed to put in extra effort to push forward. If students were really not capable of meeting the minimum milestones, I would work out a separate plan with them individually so they still had a milestone to work toward.

Personalized Curriculum

For all the challenges involved in the roll-out, the initial benefits of blended learning were substantial. Instead of spending hours each week correcting homework assignments, students were receiving feedback on their answers in real-time with no human grading required. They were also able to access hints and videos while completing assignments at home. I was able to understand and target gaps in students’ foundational understanding and to provide personalized learning paths for all of my students. The improvement in achievement and engagement was so stark, I decided to take the leap and drop the textbook entirely for the coming year – we were rolling out a new 1:1 iPad program for 5th grade, and I took the opportunity to rely on Khan as the backbone of the curriculum.

Ditching the textbook was a big change, and not without its challenges. I started to realize, though, that the content available through Khan Academy offered everything a text offered and more. Some students had begun approaching new problem sets by pressing the ‘hint’ button again and again until they had shown all the hints available – the result looked just like a page out of a textbook – even without videos, students were able to learn new topics without being taught through direct instruction.

Impact of blended learning initiative on Grade 5 2015 mean ERB scores
Impact of blended learning initiative on Grade 5 2015 mean ERB scores

Others preferred to dive right in to new topics by combining prior knowledge and trial-and-error. When I compared the online way to the textbook, it seemed like the only thing missing was the paper. Using Khan as an in-class teaching tool as opposed to just a homework tool helped me to better understand the process students were using to approach new concepts or practice old ones. These observations eventually became part of our established routines, and the instructions I gave to incoming students the following year.

The first issue that jumped out in students’ processes was that many students were not writing out their steps when solving problems online. Khan has a built in ‘scratchpad’ feature, but it’s just not the same as writing on paper; aside from the fact that it is hard to write legibly with a finger and the space is too small, the work disappears when a student moves to the next problem so there is nothing for me to refer to when a student has difficulty. Reminding students to write out their work did not have enough of an impact, so I began including written work as part of the homework/classwork grade: I still let them decide whether a given problem could be done in their head, but they needed to at least list the problem sets they had completed and the time spent on each one. I provided each student with a graph paper math journal and checked-up regularly on their entries.

Making a smooth transition to blended learning, though, took more than just getting students to keep journal entries. I was fortunate to have a principal who was very supportive of technology and innovation, but these new methods raised a whole host of concerns for parents and some administrators. I had not fully realized the technophobia present in a place as modern as New York City – some parents insisted that their children ‘couldn’t learn from computers,’ or seemed to think that students spent their entire math periods staring, zombie-like, at computer screens while I hung out at my desk drinking coffee with my feet up. Others were so accustomed to a yearly program (September is whole numbers, October is fractions, etc, etc) that the idea that students would be learning different topics at different times just didn’t compute.

Building Consensus

In my next entry, I’ll explain how the program evolved both to improve the quality of instruction and to better communicate the value add to all stakeholders. By adding a leveled grouping component, students were better able to learn and problem-solve collaboratively – anyone observing a class could easily see students’ enthusiasm and their engagement with both the content and with their peers. Scheduling class visits for parents and co-workers allowed others to appreciate how technology wasn’t an end in itself but part of a larger process. I also began posting videos (both instructional and in-class footage) and other resources online to help parents understand how Khan worked and to see how it was integrated into class. Some parents felt left-in-the-dark because they couldn’t look up the day’s lesson in the text and re-teach it to their children as they “helped” them with their homework, so I began finding ways that parents could access instruction online (I still think such a model is problematic, as it assumes students should be learning at home from tutors and parents rather than encouraging teachers and families to find ways for students to learn while in school, but that’s a story for another day).

Impact of blended learning initiative on Grade 5 2015 mean ERB scores
Impact of blended learning initiative on Grade 5 2015 mean ERB scores

Fortunately, as time went on, the benefits of the program became clearer to more of the community at-large. Families that were most resistant began to come around once they saw the progress their children were making. Struggling students were able to move quickly through remedial content, and advanced students were challenged in a way they had not been before. When test scores came back showing double-digit percentile increases in mean and median scores, I was ecstatic. Of 26 students, 13 improved by over 20 percentile points in either reasoning or computation. One student’s reasoning score jumped from the 20th to the 90th percentile. Though there is a lot to be learned outside of test scores, they were one more way to show that the risk was paying off.



What do you think?

Do you have a story to share about implementing a blended learning program? Are you looking to gamify your classes? Please share your thoughts or tell us about your personal experience with blended and gamified learning. Contact us.