Activity Prototypes

About Practice Exercises

Practice exercises with detailed feedback are an essential tool for independent math study. They provide the opportunity for students to apply what they’ve learned, identify potential problems, and rehearse their learning which in turn supports memory of math facts. Our proof-of-concept learning resources include practice exercises: a factor tree and an exponents matching exercise. These exercises are redesigns of existing online learning resources that we tested in our usability lab. Our redesigns address some of the problems we found, including:

  • Lack of feedback on submission
    • Accuracy of answers
    • Hints to address mistakes
  • Small problem sets (the user is unable to try again)
  • Confusing hints and error messages, when they were provided

About Prototypes

Website prototypes are used to work through design questions and functionality needs before programming begins. Prototypes can be created with varying levels of detail. We wanted our prototypes to describe all possible interactions between the user and the algebra practice exercise. Creating the prototypes is a discovery process, where you realize the interactions for which you need a plan by running through scenarios of using the exercise (e.g. What if they decide to change the number at the top of the tree mid-way through? Should this be possible? What will happen when it is changed?) Design patterns and usability guidelines can help answer these questions (e.g. The guideline “Give the user as much control as possible” indicates that they should indeed be able to change their answer). It is far less expensive to change design at an early stage than after programming.

Fully scoped prototypes are a powerful communication tool for use by the interaction designers and the programmers. They are useful in the RFP stage, providing developers with a detailed picture of the work involved, so their proposals can be realistic. During development, should functionality not match the interaction designer’s intentions, the relevant area of the prototype can be used as a base for discussion of the actual desired behaviors.

Two students with learning disabilities participated in iterating the prototype design. The students had previously participated in analyzing the initial usability evaluation data and in creating the alignment drawing, so they were familiar with the problems we needed to address. These students received Research Experiences for Undergraduates (REU) funding through the NSF-funded STARS Alliance as part of the Broadening Participation in Computing program.

Our Process

We started with pencil-and-paper sketches to get the basic outline of the exercise. Two of us came up with sketches for the same exercise and we all evaluated which elements of each we wanted to include. We then used Adobe Fireworks to take the sketches and create clickable pages. The prototype did not allow for actual entering of numbers, but shows what happens when you click a button or enter correct or incorrect information. The pages include textual notes to the developer to indicate details about the current page. We posted these mock-ups online and solicited feedback from our advisory panel and our internal team. The prototypes were refined based on the feedback we received.

Factoring Exercise

Factor Tree Exercise Prototype

The factor tree exercise was developed by an external programmer, Ken Richard, who was not aware of our usability evaluations and does not have a background in learning disabilities. Our prototype needed to include instructions for all possible scenarios in the factor tree exercise. We used a combination of clickable prototype screens that contain developer notes and a list of development guidelines to help ensure the planned behaviors were incorporated.

In the long hours we spent iterating through the prototype, there were a few times where the students debated as to why upfront analysis of desired user interactions is worth the time investment. However, when we made important changes after a good night’s sleep, and as we received increasing positive feedback from team members, it became clear that our work would make a difference to the success of the final learning resources. Our programmer was able to complete the work within our requested brief timeline. This would not have been possible if we had many missing interaction descriptions in our prototype.

Lessons Learned

It is often the case that one misses a couple of scenarios and design guidelines in the prototyping phase. Some of the lessons we learned include:

  • Schedule weekly check-ins with the programmer. These were very helpful for identifying problems before they became too costly (in terms of time) to fix.
  • Include significant testing time and resources. It is important to have more than one person test out a system or activity, as they will have different approaches to using the resource.
  • Identify the level of control needed over the programming methods. While we specified: “The activity should be coded using valid, standards-based html, css, javascript, and php. Exercise must be fully accessible with js implemented as a progressive enhancement,” there were elements that met this specification but we might have preferred to be done in a different manner:
    • Identify in the prototype what should and should not be controlled by javascript.
    • Identify what programmatic methods for using javascript as a progressive enhancement.