6.3User Trials
A trial pack was created containing everything that was needed to test the application. Three groups of users were involved in the trials: several Computer Science lecturers at the University of Stirling, the NLS map department in Edinburgh and a local history group. These groups of testers brought a range of abilities across a wide range of skill sets to the application testing. The first group mentioned would have experience and knowledge of computing while the second group would have experience of maps and the last group would be experienced in the historical aspects of the application. It was the last group mentioned which would be of most interest as they were in their older years, keen to use the application but had little experience of mobile devices (and indeed of computers in general); the ideal subject to test both the usability and accessibility of the application.
An important part of the testing was to find out how easy it was for users to use the application, giving them a “trial” pack with a questionnaire was considered to be a good way to achieve this [72]. Although most testers were not given a demonstration of the application, a user manual was supplied in the trial pack in case one was needed.
To get advice about the type of questions to ask in the questionnaire, Kate Howie of Stirling University was contacted. She recommended adding in some demographic information such as the age and gender of the tester. One of the aims of the questionnaire was to find out how successfully the design goals of the project were met. Creating a long questionnaire would possibly have put people off trialling the software so the number of questions was limited to ten and covered the main goals of the project:
-
How easy the software was to use
-
The usefulness of the accessibility features
-
The reliability of the application
-
The responsiveness of the application
The questionnaire created is shown in appendix 1.
6.4Feedback and Results
Twelve questionnaires were completed in total. One aspect that was surprising was the quality of feedback from many of the reviewers; comments were interesting and several helpful suggestions were made about the user interface and new features to add.
6.4.1Was the software easy to use
All of the testers said the software was easy to use. Interestingly, some feedback indicated that several Android features were confusing and not the application itself. One tester thought it was inconsistent that on the last content page you had to press the “Back” key rather than swiping the screen to go back to the content list (but in Android, this was the standard way of going back to the previous activity). Another tester correctly pointed out that if you were unfamiliar with the phone then you may be unfamiliar with the interface although they added that it was “pretty intuitive”. One tester noted that even though they were unfamiliar with mobile phones they found it easy to use.
6.4.2Accessibility features
When asked which of the accessibility features they used, one tester from the history group wrote that they used the larger text size and that was something they liked. Unfortunately, several said they were not sure how to access this feature. This seemed to indicate that some users were not familiar with the Android menu button and so they may have missed several features of the application. To avoid this happening, a late change was made to the application to add a “Settings” item to the contents list where it could be easily seen by users (as shown in Figure 9.).
6.4.3The best features of the application
When asked what the best features of the application were one mentioned the aerial view of Edinburgh and that when places were selected on the map the first page to appear showed an old image of the location. The explanations and help sections were also praised as being useful. The fact that no Internet connection was required was cited by one tester as being an important factor for a multimedia application (one tester liked this feature since they had no computer and no interest in getting one!). The easy to use navigation was cited by one tester as the best aspect of the application and the use of the maps and the map flyover was also mentioned.
6.4.4The worst features of the application
Several testers said changing the maps by pressing the map title was not obvious; a possible solution to this could be to add a button next to the title in the map page. One reviewer thought that adding a caption to the engravings and maps would be helpful as well.
Early in the testing, several changes were made due to user feedback. Many of the testers found the operation of the front screen confusing and so (as mentioned previously) the touch actions were modified. An additional change was also made to the map page; to give a visual indication of when the application knew the users’ current location, the title of the map was underlined.
Several testers commented on the orientation issue; one said switching between landscape and portrait screens was a little tricky and it would be easier if only one orientation was used. Another reviewer said the lack of a portrait mode for the pages was confusing at times and that the application should support both orientations seamlessly (this change was then subsequently made). Additionally, zooming the map was sometimes a bit “unpredictable” on occasion (as mentioned earlier, it was believed that this issue was a fault with the device itself as other applications on the smartphone were seen to do this as well). As was seen in development, one tester said the swiping was too sensitive when moving between content pages. One reviewer said the user manual could be improved and that the buttons were too small (upon further questioning she said it was the buttons on the device and not the ones in the application)!
When asked what features they would like to see, several wanted the ability to interact further with the content; to add their own notes and geo-tagged images and even to be able to add their own new places and content. Also cited was the ability to download new content as it was created and to add in “Augmented Reality” (discussed further in section 7.2). Another mentioned adding in the ability to use multi-touch screen gestures (pinching in and out) for zooming into and out of the maps.
The two ideas suggested in the questionnaire of adding the ability of take photographs and changing the colour of the pin markers when a site was visited were both seen by some reviewers as worthwhile additions to make. Another suggestion was to include contemporary photos of the historic buildings (to show what building was in its place if the original building was demolished). Another suggestion was to add the ability to close the dialog boxes by tapping them saying it would be more intuitive than using the “Back” key.
Share with your friends: |