Microsoft Word hci-notes-11 doc
description), and making detailed analysis of recordings in order to understand the patterns and rules of the situation (work analysis
Download 189.31 Kb. View original pdf
|
- Navigate this page:
- Lecture 8: Usability evaluation methods Summative Evaluation techniques Summative
| description), and making detailed analysis of recordings in order to understand the patterns and rules of the situation (work analysis). Chapter 13 of the Carroll book (by Graham Button) gives a more detailed introduction to ethnomethodological studies of work. Methods for analyzing qualitative data Despite the name ‘ethnomethodology’, Suchman’s work offers a theoretical critique, rather than a methodology for design (the word refers to the study of ethno-methods, that is, contingent and accountable patterns of human interaction, and has nothing to do with any technical concern. There are some rigorous approaches to analysis of video data, oriented toward these concerns, and used by Suchman and others at Xerox. However, these require specialist training and are probably too time consuming for routine design work. In cases where it is 36 necessary to take an open-minded approach to qualitative data such as interview transcripts or think-aloud protocols, many HCI researchers use a technique known as grounded theory, in which individual statements are coded and categorized in response to the data itself, rather than framed by a prior hypothesis or assumptions about how users ought to interact. Chapter 7 of the Cairns and Cox book gives a good introduction to grounded theory. 37 Lecture 8: Usability evaluation methods Summative Evaluation techniques Summative evaluation, often performed under the umbrella of usability testing is carried out at the end of a project after the system has been built, to assess whether it meets its specification, or whether a project was successful. This is in contrast to formative evaluation, where the main objective is to contribute to the design of the product, by assessing specifications or prototypes before the system has been built. Formative evaluation is often analytic (it proceeds by reasoning about the design, while summative evaluation is often empirical (it proceeds by making observations or measurements. The evaluation carried out inmost Part II projects is summative in nature. Summative evaluation is also used frequently in research situations, where the performance of anew interaction technique is assessed for scientific publication. However, summative evaluation is not so popular in commercial settings as in academic settings. After a system has been built, the creators tend not to be interested in further advice – many small companies consider that releasing a product is so cheap that they might as well release it as test it. Any usability problems can be resolved inversion, in response to user feedback. An alternative is to use discount usability techniques that are less rigorous than academic studies, but still give more information than crossing your fingers and hoping that users will like it (although that is a surprisingly common approach in small companies. Larger, more established, companies spend more on summative evaluation of new products, because of the danger to their reputation if they were to release a product that was very much inferior. For this reason, companies like Microsoft carryout summative evaluation studies of all products, before they undergo even early (beta) market release. Usability problems can then be tracked and resolved in the same way as other software defects, using the same process as for functional bugs found during system testing. Download 189.31 Kb. Share with your friends:
|