Design tool preference in the real world

Again, due to the impracticalities and purpose of the study, no statistical analysis was conducted. In comparison to design tool preference during the experiment, design tool preference in the real world was more diverse and variable as subjects were free to indicate different tools they may prefer to use if they were in real world design situations. This also points out that different individuals may be more/less experienced with and exposed to different tools, and whether they have access to it to increase experience is also a factor that could affect an individual’s preference for a design tool(s). In real world design situations, subjects indicated that they would prefer using: paper and pen; tablet PC (InkKit); computer (other computer applications); pen and paper and tablet PC; pen and paper, and computer (other computer applications), and one subject also indicated no preference for different design tools.

Particularly, with the finding that most participants would prefer using both pen and paper and (then) tablet PC, results supports and confirm previous findings that conventionally, professionals (and students) often design with pen and paper first, then move onto using computer to make the design look more formal (i.e. beautification). This also reflects design education where students are taught to use paper and pen first, and computer applications second. Furthermore, one is constantly reminded of the advantages of using pen and paper, for example, during planning; essay plan; art; mathematics; architecture; graphics design; engineering etc, by different disciplines in the education sector (e.g in the context of human-computer interaction, Beudouin-Lafon & Mackay, 2001; Goel, 1996; Helander, 1988; Laurel & Mountford, 1990). Such teachings are derived from the findings of many studies that suggest the sole use of computer as being disruptive to the natural design process (e.g. Black, 1990; Goel, 1996) and may affect the design outcome (e.g. Black, 1990; Goel, 1996); and the use of paper as being a greater tool for sketching – one of the most important design activities during the early stages of the design process (Black, 1990; Goel, 1996; Goldschmidt, 1991, 1994; 2003, Tversky, 1999; Van der Lugt, 2005). However, in spite of the emphasis of and findings supporting the advantages of using paper and pen in design (especially during early stages), due to the resource constraints such as time and efficiency of transferring a paper design onto a computer, (student) designers often choose not to use paper, but to design using a computer at the early stages of the design process and some may use a computer from the very beginning. For example, Newman et al. (2003) found in their web design practice study that designers often move, from working with pen and paper, onto working on the computer earlier than they wish to, but due to time constraints, they had to.

Additionally, preference in the “real world” could also be affect by the nature of the design task – whether early stages or the refinement stages in the design process or whether it is simple or complex design task, for example, simple tasks where not much planning is needed (especially for expert designers, for example, designing a simple online form that requires only the users’ name, address and a submit button) verses complex designs where planning of website architecture, information design, and individual pages’ content and layout etc, maybe needed. In addition to factors such as familiarity and competency, exposure and access to design tools, and whether a new design tool has better usability, learnability, and functionality; after all, in a sense, it might just crumbles down to personal preference for which tool to use – no matter how good a mobile phone is with all the functions possible such as digital cameras, recorders, calculators etc that clearly has its advantages, one might still choose to use the conventional mobile phone where making and receiving phone calls, texting and receiving texts is good enough, and will do the job.

4.7.1. Implications on sketch-based tool development: Recommendations for InkKit

This study also served as an informal, preliminary usability analysis of Inkit. Ultimately, InkKit has the potential to become an “all-in-one” design tool that supports sketch-based input; while providing sketch editing functions (e.g. select and move, relocation, resize, deletion, etc) as well as beautification capabilities (e.g. alignment, line smoothing, standardization etc). Moreover, it is possible that in the future, Inkit could support exporting of high-fidelity (high-formality) prototypes, in just one click of a button, whether as a word document, excel, or Visual Basic.Net, or as HTML (as ink input data is stored as XML files); allowing a prototype to be worked on using different applications. The current version of InkKit, although still in the development stage, is showing some promising results, where a story board, consisting of pages of hand-drawn sketches connected to each other in the desired way, can be exported as an HTML page, where the designer can navigate through it as if he/she was using it on the internet.

Tthere is also an implication on the types of beautification functionalities to be included in sketching-based design applications and also, if beautification is used – at what stages would it be most useful and effective? It could be useful to have different levels of beautification to produce prototypes with different levels of formality for different purposes – for presentation, review, or to serve as a display of layout. Future development of beautification functionalities could also involve colour combinations, HTML look-and-feel, shadows for buttons, and textboxes to allow input for testing and evaluation. Furthermore, this means that if the preliminary design is primarily for internal use, designers may want to skip step the using higher levels of beautification and use low to medium levels of beautification to just to slightly tidy up a designer, but when presenting a design to clients or a review committee, then higher levels of beautification may be useful.

From observations and comments expressed by participants, there are some recommendations for Inkit, in terms of usability improvement and technical development:

1) Speed of moving ink strokes around – sometimes very slow when moving or resizing after a while of working on a design – buffer problem for moving data? It is likely in applications such as Inkit, that a user would sketch and manipulate the content frequently; therefore, one of the future development of Inkit should be on improving its support for vast amounts of data manipulation – so that the simultaneous feedback can be achieved.

2) One of the many difficulties faced in this study was the programming of beautification functions in Inkit and one of them being hand-writing beautification – no methods exists for mapping text in computerized fonts into ink strokes, which could be manipulated as an ink stroke, as if it was an ink input on the screen. Hence, the back-door method was used to creating different font that appeared to be more or less formal objectively, by producing the text in the design as “labels” (programmed in Inkit to show at the desired location on the designs). Future development of Inkit could also look at overcoming such problem by

3) The previous point leads to the subject of reverse engineering where one can take a webpage, and apply reverse engineering methods, to “de-beautify” the web page and end up with a less formal design, for example, becomes mono-coloured with only black as the ink colour (like the designs in the study). However, what level of formality a webpage should be strip down to is still the fundamental question yet to be answered by future research. Reverse engineering could be useful for redesign / review purposes as suggested by the findings from the current study – more changes made (hence, higher proportion of quality changes i.e. improvements) in designs with lower formality compared to high formality designs; people may not think a web page needs to be changed as it looks formal and nice already, but maybe the prettiness has blinded the designer from usability problems etc.

4) Changing of modes.. – practice effects? Maybe a tablet rubber? Like a pen, pencil and rubber interaction? Will it be the same? What’s are the effects? How will it compare with the pen tap to change modes?
There are also implications on “rapid beautification” (during sketching) and “smart beautification” (during formalization-time after sketching). Rapid beautification, if inappropriately implemented and used, could interrupt the design process itself which is, in turn, destructs the main purpose of beautification in informal tools. This interruption could also induce annoyance and frustration when unnecessary and/or unwanted beautification appears. The findings from the present study suggest that rapid (immediate) recognition and beatification may have a negative effect in the design process. Furthermore, even with beautification at formalization time, as a design appear prettier, and more formal, fewer changes would be made to a design – even during the early stages of the design process, in which informal properties of designs have shown to be important in this study; thus, it may jeopardize the continuous, iterative process of and activities during design, and ultimately, the design quality of the end product (starting from early stages of the design process as informal design tools are used during that stage). As Plimmer and Grundy (2005) agreed, developing sketching-based design tools that provide different type of beautification is not straightforward. Therefore, more research is required in the future to explore the effectiveness of using different beautification techniques at different times in the design process, and design tasks in different domains
4.7.2. Improvements in the design process

Beautification in sketching-based design tools bridges the gap between paper and pen and the computer, thus, the conventional transferring process of the design from paper to the computer (a beautification process itself) is eliminated. This, in turn, could result in an increase in efficiency (and also motivation?), which means more time for other design activities – as designers can concentrate on sketching and ‘designing’, and beautify deigns in just a few moments.

There is another implication from such findings on the design process in practical work situations. As Newman et al. (2003) found in their web design practices study, in practical design settings, particularly for client presentation – designers often want to present to their client a professional looking design; thus, designers have to transfer their paper design onto the computer by recreating the designs from scratch in common computer applications such as photoshop. However, as Brink et al. (2002) suggested, (also confirmed in Newman et al.’s (2003) study) one frustrating problem with client presentation in the early stages of the design process is often the level of formality of the design to be presented – designers want their designs to appear professional, yet not wanting the client to view it as a finished design. With tools currently used in the professional domain (i.e. pen and paper and computer software), to create designs that appear unfinished and conceptual (informal) yet looking professional, medium-levels of beautification could be useful in this sense to create designs that appear not too rough (hand-drawn, sketchy and untidy) or too formal (computer-rendered, finished-looking designs created by computer applications such as Photoshop, Word, Publisher etc.).

Generally, in the context of beautification in early stages of the design process, as formality increased, people noticed fewer ‘errors’ in the designs. There are implications for the design process and the final product – the use of beautification should be considered carefully, especially during the early stages of the design process when exploration of ideas occurred, as the results form this study suggest that the more a design is beautified, the fewer changes people make to correct design errors. Hence, it is costly for design companies if ‘errors’ are noticed late in the production phases

There are implications from the results of the study, particularly the effects of formality on quality and expected changes, on design education that are inline with the implications from design education research – that students should be taught to design (creation, review and edit) with paper and pen first during early stages of design, as opposed to using a computer, because paper and pen supports sketching activities which have been shown to be one of the most important design activities during the early stages of the design process (e.g. Bilda, Gero, & Purcell, 2006; Goel, 1996; Lugt, 2005; McGown, Green, & Rodgers, 1998; Ullman, Wood, & Graig, 1990; Verstijnen, et al, 1998).

Likewise, with sketch-based interfaces becoming more popular as pointed out by Pomm and Werlen (2004), along with beautification (and recognition) techniques being a explored (Qin, Wright & Jordanov, 2000; Plimmer & Grundy, 2005; Pomm & Werlen, 2004; Shesh & Chen, 2004), eventually, it is likely that such tools would be accessible for design students (as well as design educators and professionals) from different disciplines – only if and when usability testing shows good results which further facilitates commercialization of such products for a wider user population. Hence, should sketch-based design tools becomes one of the main stream design tools along with paper and pen, and computers (using applications such as Computer-Aided Design (CAD), Microsoft Photoshop, Fireworks etc), the effects of formality (beautification) should be emphasize through design education, to help minimize the likelihood of error-filled designs lasting through to the later stages in the design process, for example, in domains such as web interface design; flow diagrams in mechanical design; and floor plans in architectural design. It may not be too much of a problem in design projects and assignments for students, but in the design industry, such mistake could be exceedingly costly for the design company responsible. The impact could be even more profound in the design of complex systems where errors/mistakes that are unnoticed until the later stages in the design cycle (depending on its severity) could ultimately put the design at risk of being redeveloped – i.e. back to the early stages of design; which could mean extra stress for the designers and his/her company, as well as critical strains on the already-limited resources, i.e. mainly time (and money – such as in physical product design when prototypes are required). Hence, design education and training must address the effects of formality on design performance, and thus, increase caution when beautifying designs.

Although, the findings from this study suggests that an active approach should be taken towards addressing the issues with levels of beautification and formality in sketch-based design tools in the design process – i.e. minimizing possible undesired effects of formality in early design – there are other factors that may affect the use of beautification and its effectiveness. For example: the nature and scope of the design task – whether it is a complex design (e.g. a simple website with three to four pages or a whole information system) or a simple design (e.g. a simple online form on a design that only requires a user to fill in their name and address, or a complex online form that requires more extensive user input); and the level(s) of formality one wants to achieve; here, recommendations for different situations regarding the use of beautification could be useful.

4.8. Methodological issues and limitations

In addition to the limitations discussed earlier including: small and heterogeneous sample; content in the five design; possible experimenter’s bias in the five forms designed; and levels of formality created and their intervals; the other limitation of this study was that InkKit was (and is) still in its development stage. Although it closely matches the Tablet Diary (Tablet Edition, 2005) program in which users are able to write/draw anything (using the pen), several participants commented that InkKit was “a bit hard to use” in the experiment as they had to switch between different modes i.e. Inking mode, Eraser mode, Selection mode by pointing the pen tip to the icons at the top of the screen. According to Fitts Law (with respect to VDU display design), the larger the distance from one object to another on the screen, the longer it takes for the eyes to fixate from one object to another object on the screen. This suggests that, as the hand movement is guided by the eyes, the time it takes for the hand to guide the cursor from one object to another also increases. Hence, the design task in the experiment required participants to make changes to an existing design, by deleting, adding, resizing, relocating etc, therefore, participants had less freedom to stay in one mode. This mode switching may have affected participants’ willingness to make changes; hence, results on the effects of formality on the number of changes made (design performance) should be interpreted with caution. An implication of this on interface design is to better accommodate switching of modes to support natural human-computer interaction.
4.9. Future research and directions

This study is one of the first to examine the different levels of formality and their effects on design performance, and thus, the findings from the present study raised many specific as well as wider questions on the effects of formality on design performance; which in turn, forms the basis for future direction in similar research. The limitations and methodological issues found in the present study also provide useful resources for improvements in methodological aspects in similar research in the future. In addition to the research recommendations discussed earlier along with the limitations of the study, for example, exploring more dimension and levels of formality, future research could also explore the following.

More laboratory studies are needed to further explore the effects of sketch-based design tools, particularly the effects of levels of formality of designs (as a result of different degrees of beautification), on the design performance, behaviour, cognition and perception during the design process. For example, more levels of beautification could be explored by including aspects such as the HTML ‘look and feel’ (e.g. of dropdown menus, textboxes, buttons, radio buttons); simple colours to represent differences between components (Brinck, et al., 2002); hence, whether the negative linear relationship (levels of formality and design performance) will still exist becomes an important question to be answered. Field studies should also be done to check external validity to support the (non)practicality and usefulness of beautification in real world situations such as presentation to clients in early stages in the design process (Newman, et al, 2003).

As beautification has been regarded as an important aspect of informal sketching-based tools (Plimmer and Apperley, 2004), different levels of beautification was explored in the present study. It would be useful, in terms of design education, for future research to further explore the effects of different degrees of beautification at other stages in the design process. In addition, evaluation studies on informal sketching-based tools that provide beautification functionalities might help with design tool development. Such studies will evaluate whether beautification functions can be used effectively, and if included in an informal sketching-based tool, at what level and to what extent should it be; and whether it is plausible or practical? Furthermore, it would also be valuable to examine the effects and usefulness of beautification in other (different) types of applications e.g. programs that support pen-input of music scores, mathematic formulas, simple drawings, and even kids drawing program (which has been an interest to many researchers since the sketch-based applications became popular). In addition to different applications, the effects and the effectiveness of beautification at different levels for different groups of users could also be investigated. Moreover, one question yet to be answered is whether there is an optimal level of beautification to create the appropriate level of formality of designs; hence, future extensions to the present study to provide a more comprehensive picture of relationships between different levels of beautification, formality and design performance.

The present study examined individual performance in a laboratory setting only; however, in real design situations, designers often work in teams (Stempfle & Badke-Schaub, 2002). Thus, the present study could be expanded to examine the effects of formality on team performance. However, results from such study may be difficult to interpret and examine as variability within a team may differ between teams – a combination of qualitative and quantitative data maybe useful for a fuller account and analyzes.

One underlying (and fundamental) question raised from the present study, besides the effects of formality (as a result of beautification) on design performance, is the effects of using different design mediums. Therefore, future research should explore the effects of formality on the design process using different design medium, in a more systematic manner, for example, by comparing subjects between two groups (with similar design and study backgrounds) where subjects in one group works on designs with different levels of formality presented on the tablet PC; while subjects in the other group works on designs with different levels of formality presented on paper (printed versions of the designs on the tablet PC). Thus, differences and/or similarities of using different design mediums combined with the effects formality could be explored.

Results from the present study suggest that individual differences played a role in affecting the overall number of changes made across designs with different levels of formality. It was observed that some subjects just make more changes overall in comparison to others – maybe they are more motivated to make changes. Some subjects tries hard and some subjects did not seem to exert too much effort – shown in the consistent high number of changes compared the consistently low number of changes as seen in individuals regardless to his/her design experience, study level or study major/specialization. It was observed also, that some participants were able to concentrate the whole way throughout the five conditions, utilizing every minute during the tasks as opposed to other participants who made few changes and stopped before the 10minutes time. This was also noticeable in the recorded on-screen actions throughout the five conditions – some participants had little on-screen actions compared to others who’s screen constantly changes; hence, the more changes on a screen (e.g. movements) the larger the video file. Such observations supports that motivation and personality play a significant role in influencing (work) performance, especially in the field of work, industrial and organizational psychology as shown in many psychological studies reported by Spector (2006). Future research on motivation and personality in the context of design performance will contribute to a greater understanding of designers’ interaction with different elements and artifacts within the design process – a long term research direction. Furthermore, findings from such studies could be useful to the design industry, for purposes such as personality screening during the recruitment process and motivation increase to help improve performance (e.g. design quality, design decisions, innovation and efficiency).

Chapter 5. Summary and Conclusion

Beautification was explored in the present study, by developing a taxonomy of degrees of beautification, which was validated by producing designs that appeared more, or less formal (i.e. with different levels of formality). Furthermore, the designs produced by systematically varying the degree of beautification, were also used for exploring the effects of formality on design performance – measured in terms of number of changes made to a design presented – during the early stages of the design process.

The findings in the present study confirmed previous findings by Black (1990) and Plimmer and Apperley (2004) that reviewers/designers interact differently with designs that appears untidy, sketchy, rough and informal (i.e. low formality designs) and designs that appears tidy, computer-rendered, and formal (i.e. high formality designs). In addition, the current study examined not only the two levels of formality that Black and Plimmer and Apperley looked at, but also the other levels of formality in between – where a design appeared more or less formal; hence, significant negative linear trend found.

Results showed that formality of a design affects design performance, such that as the level of formality increases, the number of changes made (total, quality and expected changes) decreases, and vice versa; demonstrating a negative linear relationship between formality and design performance. Moreover, results showed that formality of designs affected both the experts and novices, and that experts performed at a higher level in comparison to novices’ performance (demonstrated by the significant between-subjects effects). Subjective measurements included overall enjoyment and design tool preferences. Subjects enjoyed working on designs that appeared more formal (higher formality – i.e. more beautified) more than designs that appeared rougher and less formal (lower formality – i.e. less beautified). There was no difference found in the preference between designing on paper compared to designing on the tablet PC (InkKit) during the experiment. On the other hand, results showed that design tool preference(s) in real world design situations was more diverse than design medium/tool preference during the experiment.

Important implications arose from this study include: 1) design education on the effects of formality as a result of beautification, and the caution one should take when using beautification functions in informal sketching-based tools such as InkKit; 2) improvements on the design process such as easier preparation for client presentation and improved efficiency which could leave more time for actual ‘designing’; and 3) informal sketching-based tool development, in particular InkKit, to support more satisfying, natural designer-design tool interaction.

As beautification has been regarded as an important aspect of informal sketching-based tools (Plimmer and Apperley, 2004), and that designs can be beautified to different extents, thus appearing more or less formal; one fundamental question that arise, is whether there is an optimal level of beautification to create the appropriate level of formality of designs – this is particularly important to understand for designers in practical settings as client presentation is often frustrating; as one of the participants in Newman et al.’s (2003) study as a design process itself. Hence, future replications and extensions to the present study will provide a more comprehensive model of relationships between different levels of beautification, formality and design performance. This, in turn, will help nurture our understanding on designers’ interaction with informal sketch-based tools in the context of beautification; hence, a new path and angle towards design process research.

References

Bilda, Z., Gero, J. S., & Purcell, T. (2006). To sketch or not to sketch? That is the question. Design Studies, In Press, Corrected Proof.

Goldschmidt, G. (1997). Capturing indeterminism: representation in the design problem space. Design Studies, 18(4), 441–445.

Hong, J. I., & Landay, J. A. (2006). SATIN: a toolkit for informal ink-based applications. Paper presented at the International Conference on Computer Graphics and Interactive Techniques, Boston, Massachusetts.

Purcell, A. T., & Gero, J. S. (1998). Drawings and the design process: A review of protocol studies in design and other disciplines and related research in cognitive psychology. Design Studies, 19(4), 389-430.

Shesh, A., & Chen, B. (2004). SMARTPAPER: An interactive and user friendly sketching system. Eurographics, 23(3), 301-310.

Van der Veer, G. C. c. d., & Mulder, G. (Eds.). (1988). Human-Computer Interaction: Psychonomic Aspects. Berlin, Heidelberg: Springer-Gerlag.

Wong, Y.Y. (1992). Rough and ready prototypes: Lessons from graphic design. Paper presented at the Proceedings of Human Factors in Computing Systems CHI '92, Monterey, 83-84.