Over the past decade, we have been engaged in an extensive research effort to build virtual humans and applications that use them. Building a virtual human might be considered the quintessential AI problem, because it brings together many of the key features, such as autonomy, natural communication, and sophisticated reasoning and behavior, that distinguish AI systems. This article describes major virtual human systems we have built and important lessons we have learned along the way.
The Institute for Creative Technologies was founded a decade ago to bring together researchers working at the cutting edge of simulation technologies, such as computer graphics, artificial intelligence, and virtual reality to work with people from the entertainment industry who know how to create characters that are compelling and stories that are engaging to work toward the goal of creating the next generation of simulation and training systems. Early on, we decided to focus on training human-oriented skills, such as leadership, negotiation, and cultural awareness.
These skills are based on what is sometimes called tacit knowledge (Sternberg 2000), that is, knowledge that is not easily explicated or taught in a classroom setting but instead is best learned through experience. Currently, these training experiences are usually delivered through various human-to-human role-playing exercises. We sought to replace the human role players with virtual humans, which are computer-generated interactive characters that look and act like people but exist in virtual environments. There are several benefits to taking such an approach. Human-based role playing is costly in terms of personnel requirements and is often done at training centers that may be far away from the student's location. In contrast, virtual exercises can be delivered on a laptop, making them available to a student whenever and wherever they are needed, without the need to tie up additional personnel resources.
Vision for Virtual Humans
Our vision for virtual humans is that ultimately they should look and behave as much like real people as possible. Specifically, their behaviors should not be scripted, but instead they should function autonomously, reacting to events in the virtual (and real) world around them and responding appropriately. They should fully perceive their environment including both virtual and real people. They should interact in a fluid, natural way using the full repertoire of human verbal and nonverbal communication. They should model their own and others' beliefs, desires, and intentions and they should exhibit emotions. Finally, they should do all these things in a coherent, integrated fashion.
Achieving the vision for virtual humans outlined above is unquestionably ambitious. Many of the elements in the virtual human vision are consonant with the general vision for AI that McCarthy and others cast for the 1956 Dartmouth Summer Research Project on Artificial Intelligence:
The study is to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it. An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves" (reprinted in McCarthy et al. [2006]).
Are our goals too hard? It is certainly a reasonable question. Implementing a virtual human requires integrating a diverse range of AI technologies including speech recognition, natural language understanding, dialogue management, automated reasoning, speech and gesture generation, and animation. Not only are fundamental advances required in some of the subareas but also the technology for integration of the parts is quite complex.
While the ultimate goal remains ahead of us, significant progress has been made, and it is now possible to create systems that successfully use virtual humans in applications. Over the past 15 years or so, a number of research groups have pioneered efforts to explore different themes in virtual humans including body animation and control (Badler, Phillips, and Webber 1993), dialogue and nonverbal behavior (Cassell et al. 1994; Cassell et al. 2000; Hayes-Roth, Gent, and Huber 1997; Pelachaud, Badler, and Steedman 1996), and immersive training with action and dialogue (Rickel and Johnson 1999a). These led to a broad range of applications for health (Bickmore, Pfeifer, and Jack 2009; Marsella, Johnson, and LaBore 2003), interactive entertainment (Mateas and Stern 2003), training (Johnson, Rickel, and Lester 2000; Rickel and Johnson 1999b) and education in a social con- text (Johnson, Vilhj�lmsson, and Marsella 2004; Paiva, Dias, and Aylett 2005; Pelachaud et al. 2002; Zoll et al. 2006)
At the University of Southern California (USC) Institute for Creative Technologies (ICT), we have performed research to understand better how virtual humans interact with real people and how people perceive and react to virtual humans, to develop new technology that extends their capabilities, and to use them in novel applications. We have created characters to help train leadership skills (Swartout et al. 2006; Swartout et al. 2001) and negotiation tactics (Hill et al. 2006; Traum et al. 2005). We have developed virtual patients (Kenny et al. 2008) to help train clinicians in appropriate patient interviewing techniques, and we have even developed virtual humans to help with tasks such as recruitment (Artstein et al. 2009).
