GOMS Models: An Approach to Rapid Usability Evaluation

This project is a set of technology transfer activities concerned with moving the research results in human-computer interaction into practical methodologies for designing computer system interfaces that are in fact easy to learn and easy to use. The research results of interest are those from earlier and ongoing projects concerned with constructing and evaluating computational models of human cognition and performance in the context of humans interacting with systems.

The payoff of applying these models to interface design results from the limitations in the standard human factors methods for developing usable systems. These methods are effective, but are slow and costly to apply because they are based on empirical user testing: In a scientifically controlled setting, actual human users perform actual tasks using a prototype system; their performance is recorded and analyzed, along with any apparent problems and difficulties. The system design is then revised, and the system re-prototyped, and the test repeated, until overall system performance is adequate, no further problems are noted, or time and money has run out.

The goal of this work is to radically reduce the time and cost of designing usable systems through developing analytic engineering models for usability based on validated computational models of human cognition and performance. These models take a specification for a user interface design and a description of the user tasks that need to be carried out, and generate predictions of the time required to learn how to use the system, and the time required to carry out specific tasks. These predictions can be used instead of empirically collected data for much of the design process, thus saving considerable resources. The current models address the procedural quality of the interface -- the complexity, consistency, and speed of the procedures that the user must learn and execute in order to make use of the system. These models can help the design produce an interface that is reasonably usable, and then the slow and expensive empirical testing can be reserved for examining aspects of the interface not addressed by the models, and as a final check on the design.

The GOMS Model

Earlier research in HCI has resulted in a general concept, the GOMS model, which represents the procedural knowledge required to operate a system in terms of the user Goals, basic actions or Operators, Methods, which are sequences of operators that will accomplish goals, and Selection rules, which determine which method to apply to accomplish a goal. Research by Kieras and others has shown how this type of analysis can be used to obtain usefully accurate predictions of learning and execution time. This work was based on using a production-system representation of human procedural knowledge; GOMS models can be constructed using production systems, and so the empirical predictions can be generated from GOMS models.

We are involved a variety of activities to extend and apply this framework, and turn it into a teachable, standard methodology that can be applied in industry. An important first step was to encapsulate the earlier research on GOMS models into a task analysis method and model representation notation, called NGOMSL, that makes it easy to construct and apply a GOMS model. After learning this notation and techniques, software developers can calculate estimated learning and executing times, and identify man y qualitative problems in an interface design. Accumulating experience and research shows that such models are indeed practical and effective in interface design situations.

We are extending and refining the notation and modeling methodology to incorporate newer research results and make it even simpler and more useful. This notation, and the techniques associated with it, have been taught in university courses and short courses for industrial and professional groups.

Computer-based tools for GOMS models

The research shows that the GOMS model is useful and effective even when the modeling technique is done by hand. However, to be most useful in improving the usability of software, we need computerized tools that will make the model construction and application process fast and easy. Under support from DARPA, we have constructed such tools and developed them into a form ready to be transferred into industrial practice. Trials with the tool, named GLEAN, are promising, as summarized in th UIST'95 paper. GLEAN uses a formalized version of the NGOMSL notation, called GOMSL. A version of GLEAN can be downloaded, along with a Guide to performing a GOMS analysis usign GOMSL and GLEAN.

In addition, we will be developing new concepts for more advanced tools based on using the EPIC architecture together with the GOMS model, which should allow precise representation of perceptual-motor timing during procedure execution. See The EPIC Project

.

Online Materials

All downloads are pdf files unless noted otherwise.

A guide to GOMS model usability evaluation using GOMSL and GLEAN3. On-line publication describing GOMSL, the GLEAN tool, and how to perform a GOMS analysis. Supersedes the previous Guides using NGOMSL.

Downloads for GLEAN executables See A guide to GOMS model usability evaluation using GOMSL and GLEAN3 for documentation.

Using the Keystroke-Level Model to Estimate Execution Times On-line handout on an approach to using this simple form of GOMS model.

A Guide to GOMS Model Usability Evaluation using NGOMSL On-line handout using NGOMSL; similar to Kieras 1997, 1988.

Links to related projects and pages


Publications

All downloads are pdf files unless noted otherwise.

Journal Articles

John, B. E., & Kieras, D. E. (1996). Using GOMS for user interface design and evaluation: Which technique? ACM Transactions on Computer-Human Interaction, 3, 287-319.

John, B. E., & Kieras, D. E. (1996). The GOMS family of user interface analysis techniques: Comparison and contrast. ACM Transactions on Computer-Human Interaction, 3, 320-351.

Bovair, S., Kieras, D.E., & Polson, P.G. (1990). The acquisition and performance of text-editing skill: A cognitive complexity analysis. Human Computer Interaction, 5, 1-48.

Kieras, D.E., & Bovair, S. (1986). The acquisition of procedures from text: A production-system analysis of transfer of training. Journal of Memory and Language, 25, 507-524.

Kieras, D.E., & Polson, P.G. (1985). An approach to the formal analysis of user complexity. International Journal of Man-Machine Studies, 22, 365-394.

Book Chapters

Kieras, D. E. (1997). Task analysis and the design of functionality. In A. Tucker (Ed.) The Computer Science and Engineering Handbook. Boca Raton, CRC Inc. 1401-1423. preprint

Kieras, D. E. (1997). A Guide to GOMS model usability evaluation using NGOMSL. In M. Helander, T. Landauer, and P. Prabhu (Eds.), Handbook of human-computer interaction. (Second Edition). Amsterdam: North-Holland. 733-766.

Bovair, S. & Kieras, D.E. (1991). Toward a model of acquiring procedures from text. In R. Barr, M. L. Kamil, P. Mosenthal and P. D. Pearson (Eds.), Handbook of Reading Research, Vol. II, (pp. 206-229). New York: Longman.

Kieras, D.E. (1988). Towards a practical GOMS model methodology for user interface design. In M. Helander (Ed.), Handbook of Human-Computer Interaction (pp. 135-158). Amsterdam: North-Holland Elsevier.

Presentations and Talks

Kieras, D.E., Wood, S.D., Abotel, K., & Hornof, A. (1995). GLEAN: A Computer-Based Tool for Rapid GOMS Model Usability Evaluation of User Interface Designs. UISTÕ95 Proceedings of the ACM Symposium on User Interface Software and Technology. November, 1995.

Byrne, M.D., Wood, S.D, Sukaviriya, P., Foley, J.D, and Kieras, D.E. (1994). Automating Interface Evaluation. In Proceedings of CHI, 1994, Boston, MA, USA, April 24-28, 1994. New York: ACM, pp. 232-237.

Gong, R., & Kieras, D. (1994). A Validation of the GOMS Model Methodology in the Development of a Specialized, Commercial Software Application. In Proceedings of CHI, 1994, Boston, MA, USA, April 24-28, 1994. New York: ACM, pp. 351-357.

Foltz, P.W., Davies, S.E., Polson, P.G., & Kieras, D.E. (1988). Transfer between menu systems. In Proceedings of the CHI '88 Conference on Human Factors in Computing Systems, (Washington, May, 1988).

Bennett, J.L., Lorch, D.J., Kieras, D.E., & Polson, P.G. (1987). Developing a user interface technology for use in industry. In H.J. Bullinger, & B. Shackel (Eds.), Proceedings of the Second IFIP Conference on Human-Computer Interaction, Human-Computer Interaction - INTERACT '87. (Stuttgart, Federal Republic of Germany, Sept. 1-4). Elsevier Science Publishers B.V., North-Holland, 21-26.

Polson, P.G., Bovair, S., & Kieras, D. (1987). Transfer between text editors. In Proceedings of CHI and GI 1987. (Toronto, April 5-9). ACM, New York, 27-32.

Polson, P.G., & Kieras, D.E. (1985). A quantitative model of the learning and performance of text editing knowledge. In Proceedings of the CHI '85 Conference on Human Factors in Computing Systems, (San Francisco, April, 1985).

Kieras, D.E., & Bovair, S. (1984). The acquisition of procedures from text. In Proceedings of the Sixth Annual Conference of the Cognitive Science Society (Boulder, June 28-30, 1984), University of Colorado, Institute of Cognitive Science.

Technical Reports

Kieras, D.E. (1998). A guide to GOMS model usability evaluation using GOMSL and GLEAN3. (Technical Report No. 38, TR-98/ARPA-2). Ann Arbor, University of Michigan, Electrical Engineering and Computer Science Department. January 2, 1998.

Kieras, D.E. (1996). Development and exploratory applications of a GOMS modeling tool for user interface evaluation. (Technical Report No. 37, TR-96/ARPA-1). Ann Arbor, University of Michigan, Electrical Engineering and Computer Science Department. Augst 20, 1996.

John, B. E. & Kieras, D. E. (1994) The GOMS family of analysis techniques: Tools for design and evaluation. Carnegie Mellon University School of Computer Science Technical Report No. CMU-CS-94-181. Also appears as the Human-Computer Interaction Institute Technical Report No. CMU-HCII-94-106.

Bovair, S. & Kieras, D.E. (1989). Toward a model of acquiring procedures from text (Tech. Rep. No. 30, TR-89/ONR-30). Ann Arbor: University of Michigan, Technical Communication Program.

Bovair, S., Kieras, D.E., & Polson, P.G. (1988). The acquisition and performance of text editing skill: A production system analysis (Tech. Rep. No. 28). Ann Arbor: University of Michigan, Technical Communication Program.

Kieras, D.E., & Bovair, S. (1985). The acquisition of procedures from text: A production system analysis of transfer of training (Tech. Rep. No. 16, TR-85/ONR-16). Ann Arbor: University of Michigan, Technical Communication Program. (DTIC AD A151029)

Kieras, D.E., & Polson, P.G. (1982). An approach to the formal analysis of user complexity. Project on User Complexity of Devices and Systems (Working Paper No. 2). Tucson: University of Arizona, Department of Psychology. Boulder: University of Colorado, Department of Psychology.