8. User evaluation In this section we present a user evaluation of a concrete application of Supple automatically generating user interfaces adapted to the individual abilities of users with motor impairments. As we have argued earlier in the paper, there is a mismatch between the effective abilities of people with motor impairments and what the creators of typical interfaces assume about the user’s strength, dexterity, range of motion, and input devices. This mismatch can prevent or impede interaction with computers. In contrast, even users with severe impairments can effectively operate user interfaces designed with their unique abilities in mind (e.g., [31,38]). Because of a great variety in individual abilities [5,39,41,46], many such user interfaces are needed. Unlike manual redesign, automatic generation of such individual ability-based interfaces is a scalable solution. 8.1. Overview of the approach We evaluate two approaches for automatically generating user interfaces adapted to a person’s individual motor abilities. The first approach uses the Arnauld system [22] to model users subjective preferences about what user interfaces are best for them, and it relies on the factored cost function described in Section 5.1 to generate the user interfaces. The second approach uses Ability Modeler [27,28] to build a model of a person’s actual motor abilities this approach uses the cost function that allows Supple to directly optimize for the expected speed of use (Section We divided the study into two parts, performed on two separate days. During the first part, each participant interacted with Arnauld and then with the Ability Modeler. During the second part, we evaluated participants performance and satisfaction when using 9 different user interfaces 3 were baselines copied from existing software, 3 were automatically generated for each participant based on his or her preferences, and 3 were generated based on the participant’s measured abilities. 8.2. Participants Altogether, 11 participants with motor impairments (age 19–56, mean 5 female) and 6 able-bodied participants (age: 21–29, mean 3 female) recruited from the Puget Sound area took part in the study. The abilities of participants with motor impairments spanned abroad range (Table 4), and they used a variety of approaches to control their pointing devices (Fig. 28). All but one reported using a computer multiple hours a day and all reported relying on the computer for some critical aspect of their lives. 8.3. Apparatus We used an Apple MacBook Pro (2.33 GHz, 3 GB RAM) laptop for all parts of the study. Most participants came to our lab for the study and used an external Dell UltraSharp 24” display running at 1920 × 1200 resolution, but 3 of the 11 motor-impaired participants chose to conduct the experiment at an alternative location of their choosing in these cases, we used the laptop’s builtin 15” display running at the 1440 × 900 resolution. Each participant had the option of adjusting the parameters of their chosen input device (e.g., tracking speed, button functions. Additionally, we offered the participants with motor impairments the option to use any input device of their 4 Numbers were calculated using the Metrics plugin for Eclipse available at metrics.sourceforge.net and reflect all method lines in classes devoted to the interface description for each of the examples.
938 K.Z. Gajos et al. / Artificial Intelligence 174 (2010) 910–950 Table 4 Detailed information about participants with motor impairments (due to the rarity of some of the conditions, in order to preserve participant anonymity, I report participant genders and ages only in aggregate).