Iterative design and user evaluation

3.1Evaluation of Active Messenger

• 
  Does Active Messenger do what the developer wants it to do?
  
• 
  Does Active Messenger do what the user wants it to do?
  

The answer to the first question is “yes.” In the current version, the agent performs in a way the programmer wants it to. There are no known problems that keep the agent from doing what the developer supposed it to do. However, this is not enough. The agent was not built for the programmer, but for the user.

• 
  User A makes Active Messenger process around 90% of all new messages. This means, the user has set up her Clues filtering in a way so that 90% of all incoming messages fall into categories for which the agent has rules. These rules tell the agent to forward such a message in one way or the other. For user A, these 90% mean 48 messages per day in average. Because she has used the agent for 21 weeks now, the agent has processed already over 7000 messages for her.
  
• 
  User B makes Active Messenger store and forward around 38% of all incoming messages, which stands for about 50 messages per day for this specific user. She started using the agent 14 weeks ago, so Active Messenger has processed almost 5000 messages for her.
  

3.2Surgery

This project is different from many other Media Lab projects: Active Messenger was developed interactively with the help of the users. This is good because the developer has immediate feedback about if an idea or concept works and improves the overall performance of the agent. The bad thing is that it can be like “surgery on living flesh”: a very sensitive task. Each time a new version is started up, the functionality of the whole system has to be checked carefully, because each bug decreases the users’ satisfaction with the system. A working communication infrastructure is very important, and Active Messenger is the centerpiece of it, because it manages all incoming text messages.
Iterative design can be frustrating because it means, “modifying the rules during the game.” The users have new ideas and insights when using the system, which often means that code segments get obsolete. However, the current status of the project, a reliably working agent, indicates that it was the right way to go.

3.3Ease of use

The very first user of Active Messenger complained about many details. Beside minor things like “I hate upper case file and directory names,” she complained that she had to create a directory to put the configurations file in. An installation wizard should do that for her.
Furthermore, she complained about the size of the generic preference file. It is too big and has too many entries. Obviously, first time users don't want to deal with settings like “percentage read from spool file,” mainly because they don't understand the meaning. The solution would be to start Active Messenger in a “basic” mode, and then give the user the possibility to switch to an “advanced” mode, like ICQ™ does.
The preference file can be reduced to a minimum, using the defaults of the PERL script. If all comments and explanations are removed, it shrinks to a few lines, see Figure 7: Sample user preference file. In summary, there are users who want to have an extensive self-explanatory preference file, and other users who prefer a small preference file and a separate manual, perhaps in the form of a UNIX manual page.
However, there are two reasons why there are so many user options in the preference file (see Table 15: Preference file default values):

• 
  Full control over what happens. Active Messenger is a prototype, and the developer needs to adjust the parameters to find the optimal basic settings. Since the user preference file is loaded dynamically, all variables of a running Active Messenger can be modified without rebooting the agent. Rebooting is not desired because Active Messenger would lose all information of what has happened so far. Running Active Messenger for a long time is also prerequisite for testing certain features. This is related to the problem that the complexity of Active Messenger makes it very difficult to test how the agent behaves in extreme situations. It requires a lot of patience (time) and adjusting of external parameters (e.g., the Canard paging system) to force it to get in an extreme program state.
  
• 
  Different users expect it to work in different ways. Therefore, Active Messenger has to be adjusted to their expectations.
  

Directory: ~stefanm
~stefanm -> Mit media Lab Stefan Marti mit media Lab running head: Active Messenger

Download 0.67 Mb.

Share with your friends: