A glass Box Approach to Adaptive Hypermedia
Interface Evaluation and Bootstrapping of Adaptivity
Download 0.88 Mb.
|
- Navigate this page:
- Method and Subjects
- Evaluation of the Interface
- The Relevance Rules
Interface Evaluation and Bootstrapping of AdaptivityOur prototype was not in a completely stable state when we did our first (proper) evaluation of it in December 1995, (Espinoza and Höök, 1996b). Partly, we were not finished implementing parts of it and partly it was due to the problems with Java and Netscape that were being released in beta-versions. Even so, we were able to conduct a first evaluation of the interface parts of our system. The purpose was to collect information that would help us to further bootstrap the adaptive parts of the system. At the same time as we collected that information we also logged the users’ actions and posed a number of queries to them on how the perceived the interface. At the time of the study of the interface, we only had the graphs and the text frame. The interface was similar to that in Figure R on page 128 but the guide frame had not yet been implemented. Instead we showed our design with the guide frame as a picture to the subjects after their session with our system, and they all expressed a feeling that this would improve the interface. Method and SubjectsThe purpose of this evaluation was to get the users’ initial reactions to the interface and its potential usefulness. At the same time, we wanted to give them the task of telling us which explanations would be most useful and relevant to them in their work given a certain task. We wanted to get at their subjective opinions and not restrict them to the kinds of explanations or relations available in our prototype. So we decided to allow for an open interview situation rather than a controlled study. Our subjects were asked to talk aloud and comment on anything they found appealing or strange. 13 subjects participated in the study. We logged all the users’ interactions with POP onto files and we tape-recorded their comments. The subjects first got familiarised with the tool by just ”clicking around”. We pointed out the four different actions possible: clicking in the graphs, posing queries via menus, opening and closing stretch-text, and hotlist actions. They filled in a form about their background knowledge. They were then presented with different ”tasks” and asked to pick one which fit with what they knew about a particular part of the process, what they usually did, their background knowledge and work-style, etc. (like ‘reverse engineering’, ‘project planning’, ‘learning structure’, etc.). They were then asked to tell us which information about a process and an object would be most relevant given the task. In addition, some subjects performed two tasks searching for specific pieces of information. Finally, the subjects were asked to answer a set of follow-up questions on their impressions of the interface. Evaluation of the InterfaceWe had two design goals particular to the interface: that the system should be interactive, but also that it should be web-like. So we wanted to know whether our subjects would be able to understand which interactions with the web-page that were possible, and whether they could anticipate the result of those actions. First, we can observe that they all made use of the possibilities to click in the graphs and the stretchtext. Everybody used the possibility to pose a query via the menus of via hotwords a few times, although their main interaction was via the graphs. Second, when asked after the session whether they had found the ”links” and possible actions at the interface understandable, eleven said that they had no problems, while two felt that as beginners they had problems. Two subjects (out of the eleven) said that the result of a follow-up query on a hotword resulting in an insertion of an explanation into the current page should be better marked, perhaps with another colour or indented. Related to the possible actions at the interface, was whether they could make sense of how the information space was organised. On the query ”Was the information space understandable (did you get lost at any point)?”, seven subjects claimed to have no problems, while three asked for go-back functions, and two were irritated by the fact that when they did what they felt was going back to a previous page, the system would have closed some information entities (this triggered the implementation of the history). Finally, one subject had problems with the navigation, but liked the possibility to have control over the textual parts and wanted those to be the basis for navigation rather than the graphs. Concerning the graphs in the interface, the users’ were not confused by the fact that they were both used for navigation (implicitly posing queries) as well as for presenting information about the target domain. Some complained about the fact that they were designed to be quite small. In general, the impression was that the subjects would have liked to decide how big the graphs versus the text frames should be. Sometimes and for some subjects the graphs were more important, while for others the text was the main source of information. (This, by the way, conforms with our distinction between graphics lovers and graphics haters). This is why we changed the interface and place the graphs in their own window, as in Figure Q on page 126. So, in summary, it seems as though our subjects made use of the possible actions at the interface and they did not have too big problems with getting lost or understanding how to interact with the system.
Adding the project status to our system and using it as an additional source of information for the adaptive behaviour, was unfortunately not possible as the project status of projects at former Ellemtel was not accessible in any reliable form. Directory: ~kia -> papers ~kia -> Usability and fun An overview of relevant research in the hci community papers -> Comparing Two Approaches to Context: Realism and Constructivism ~kia -> B. Cahour (1), P. Salembier (1), Ch. Brassac (2), J. L. Bouraoui (1), B. Pachoud (3), P. Vermersch Download 0.88 Mb. Share with your friends:
|