Microsoft Word hci-notes-11 doc
Download 189.31 Kb. View original pdf
|
- Navigate this page:
- Bad evaluation techniques
| Field tests Some very successful software companies have carried out field testing of their products in addition to field studies at the specification phase. A well-documented example is the “follow-me-home” programme carried out by Intuit Inc. after the release of their Quicken product. Company researchers selected customers at random, when they were buying a shrink-wrapped copy of Quicken in a store. The researcher then went home with the customer in order to observe them as they read the manuals, installed the product, and used it for their home financial management. Intuit directly attribute the impressive success of the product to this type of exercise, and to the observational studies they carried out during initial product planning. (Quicken survived an assault from a Microsoft product priced at a predatory $15, and Microsoft later made a bid of $1.5 billion to buy Intuit. Intuit continues 41 to thrive as an independent company, and all members of staff, from directors to programmers, are expected to spend at least 10% of their time visiting customers to understand user needs. Chapter 14 of Preece, Rogers and Sharp discusses field studies. Bad evaluation techniques Some user interface developers use evaluation techniques that are practically useless. Unfortunately these techniques can even be found in some published research in computer science. This section is included as a warning to interpret such results with great care. Simple subjective reports seldom give useful information about interface usability. When users are shown a shiny new interface next to a tatty old one, they will often say that they like the new one better, regardless of its usability. There are many circumstances in which a person's introspective feelings about their mental performance is not a good predictor of actual performance, so this type of report is unreliable as well as open to bias. Some research proposes a usability hypothesis, then does not test it at all. It was proposed that more colours should be used in order to increase usability. This type of statement is speculation rather than science designing novel user interfaces without any kind of experimental testing is rather pointless. There is a great deal of variation between different people in their ability to use different interfaces. This may result from different mental models, different cognitive skills, different social contexts and many other factors. Any conclusions drawn from an observation of only one person must therefore be very suspect. Unfortunately, many user interfaces are developed based on observations of a single person - the programmer. The introspection of the user interface developer about his or her performance is seldom relevant to users. The word intuitive is often used in discussion of user interfaces to summarise theories based on all the above, so should be considered a danger sign, if it is used to describe the advantages of a particular user interface design, without any further specific detail or empirical evidence. Download 189.31 Kb. Share with your friends:
|