Session: Knowledge Sharing and Reuse

The Development of an Intelligent Tutoring System for Training Customer Contact Personnel

Charles P. Bloom

US WEST Technologies, USA

The Learn, Explore and Practice (LEAP) intelligent tutoring systems (ITS) platform represents the product of a two-plus year effort in technology research, development and transfer. LEAP is a general, multimedia ITS platform that provides customer contact employees (CCEs) with an intelligent, coached environment in which to practice customer contacts (i.e., conversing with customers to solve problems and sell products and services, while simultaneously interacting with a number of main-frame software applications). In LEAP, trainees can exercise their customer interaction skills by working through typical customer interactions in a tutoring environment that accurately emulates their job environment to facilitate transfer from training to work. As trainees work through contact scenarios, LEAP applies instructional strategies as appropriate for each individual trainee. In addition, trainees can request advice from LEAP at any time, and can review their performance after finishing each scenario. In addition, LEAP allows instructional designers to adjust LEAP's instructional and student modeling parameters to further individualize the delivered instruction.

Version 1 of LEAP underwent a comprehensive evaluation in the summer of 1993, the results of which have driven significant revisions to the system in order to meet a U S WEST corporate expectation that LEAP be deployed in the fall of 1994 to support training of all customer contact employees. In particular, it was felt that three primary areas in version 1 needed enhancement:

  1. LEAP used a dialogue management method that restricted all contacts to a single correct path (job analysis results indicated that there is significant flexibility on the ordering of some dialogue activities).
  2. LEAP only simulated a single application for CCEs to interaction with (CCEs often interact with as many as three or four applications simultaneously).
  3. LEAP's reflective follow-up functionality did not allow trainees to inform LEAP on how they thought they did, nor did it provide contact specific feedback.

Revisions included in Version 2 in response to these findings include:

  1. Increasing LEAP's dialogue flexibility to better support cross training and refresher training of more experienced personnel.
  2. Supporting the simultaneous presentation and interaction with multiple application simulations.
  3. Enhancing LEAP's reflective follow-up functionality to provide more instructional review options at the completion of each contact rehearsal.
  4. Conducting of traditional software engineering practices to produce a more robust and easier to maintain system, eventually to be transferred to a deployment and support organization.

An Interactive Learning Environment Using Focusing on and Perturbation of Media Representations

Kohji Itoh

Science University of Tokyo, Noda 278, Japan

According to J. Barwise and J. Etchemendy, "homomorphic representations" are representations in which the semantic structure of the represented world could be more or less directly observed in the syntactic structure of the representation. The use of such representations, e.g. diagrams, along with linguistic representations is ubiquitous (also called heterogeneous) in human reasoning. It is argued that the increase of efficiency in reasoning comes from the ease of sensory motor operations (or their stimulation) of the representation by human problem-solvers, as well as from the often observed explicitness of (sub)goals in (sub)problem descriptions. Interactive learning environment systems are proposed to provide students with building blocks which are used to construct and manipulate homomorphic and heterogeneous representations of their problems in collaboration and "agreement" with the system. They also endow the system with such abilities as numerically focusing and manipulating the same representation. Thereby, the invariants, cases, or conflicts are discovered that are to be suggested to the students.

An algorithm has been developed which enables students as well as the system to perturb their representation for preserving the already satisfied constraints in order to satisfy additional constraints or to discover invariants and cases. The environment has been implemented as an object-oriented knowledge-based system. For illustration, sample dialogue sessions that could be held between the students and the system are used. Such prototype learning environments are currently under development in several domains (e.g. elementary geometry, electromagnetics, electrical circuits, and dynamics).

Integrating General Information and Knowledge with Specific Experiences in an Environment for Teaching and Learning Assistance

Agnar Aamodt

University of Trondheim, Norway

The basic idea of this work is to combine a knowledge-based approach, based on a semantic network represen-tation of knowledge, with a hyperlink network for student/-teacher browsing of concepts, relationships and submodels. The two networks represent two views of the same basic data structure - a densely connected network of objects and relations. Hence, a platform for knowledge sharing is established, since the same data structure can be interpreted and utilized as semantic network knowledge for the system and as hypertext information for the user. Knowledge (for example partial domain models) and information may therefore be shared between computer and users, given that the computer contains inference methods that are able to interpret the network structure in a similar fashion as a human user. Correspondingly, knowledge may also be shared between several users, e.g. teachers and students. Upon this basis, we are studying methods for knowledge- intensive case-based reasoning for computer-aided instruction.

The more general research agenda that lies behind this work is the reuse of experience for intelligent decision support in open and weak theory domains. Learning from experience is an emphasized issue, since it is hard to see how future user-cooperative systems, that have to deal with increasingly complex and continuously changing application domains, can do without adaptive learning abilities. A particular topic is how a model of general knowledge can be used to focus the reuse of past experiences (i.e. cases). Another topic is how a manual utilization of past concrete experiences can be combined with automated case-based reasoning and learning. Finally, the problem of learning in the 'system as a whole', i.e. student learning as well as machine learning, is discussed within the above context. These three topics should be viewed as an approach to integration along the three dimensions, defined by the three pair of end points: specific cases / generalized knowledge, manual problem solving / automated reasoning, learning in student / learning in system.

To represent network knowledge, we use a frame-based knowledge representation system (CreekL) with default inheritance, with self-descriptive (reflective) properties, and in which relations (slots) are also represented as concepts (frames) to be explicitly modeled and reasoned about. Integrated use of different knowledge types is at the core of our methodological approach, and we are continuously looking for synergy effects within, as well as between, the three dimensions. The integration view has consequences for each of the separate methods that underlie the sharing of knowledge across humans and machines. For example, how the case indexing problem is solved when there exists a body of general knowledge and information in which the cases are integrated, and how the case reuse methods are affected by a combined manual and automated reuse of cases.

Our first goal is to develop a framework and a high level system architecture that is suitable for describing the relevant properties of these dimensions, and for analyzing their interrelations. At the basis of this framework is a distinction between information and knowledge, according to their different roles in a learning process (or a decision process in general), and whether the point of reference is the human user or a machine. The framework will, in turn, form the basis for a system design in which case-based reasoning and learning is the 'running engine' in an environment for teaching and learning assistance, and in which also more data- intensive and knowledge-poor methods will be studied and compared.

Discussion Group: The Role of Case-Based Reasoning in Intelligent Tutoring Systems


Agnar Aamodt and Peter Reimann

University of Trondheim, Norway, University of Freiburg, Germany

We find three important reasons why cases and CBR techniques begin to play an increasing role in instructional systems, ITS in particular. The first reason is a psychological one: Students - novices and beginners, but also often more advanced learners - make frequently and intensively use of case information, for instance in form of examples (worked out solutions to problems) by referring back to former problem solving episodes produced by themselves. CBR serves them both as a problem solving method and as a form of analogy-based learning: Repeated use of cases may lead to generalized knowledge structures. The second reason is a pedagogical one: Reasoning with cases is in many areas - in particular those taught at the university level - an established way of teaching: Think of the Harvard style of teaching law and business. The third reason is a pragmatic one: In those domains which are hard to formalize - such as law - or where using the 'deep' causal knowledge is too cumbersome for most practical purposes - such as medicine -, teaching with and learning from cases is often a practical way to provide computer- based instruction.

Looking at the current practice of using cases in computer-based instruction, we can distinguish four types of applications:

  1. Diagnosis of students' knowledge. A good instance for this type of application is Gerhard Weber's ELM system, that is able to come up with a planning rationale for students (LISP-) programming behavior by using a knowledge base of programming knowledge (formalized as rules). Since operator-based plan recognition is a cumbersome and computationally expensive process, ELM stores the plan structure in an episodic memory.

    When attempting further plan recognition, ELM will first look into its case memory whether it hasn't analyzed similar errors before and if so, whether the former explanation could not also account for the current programming mistake. This lookup in the case library is computationally much cheaper than full plan recognition, and has the added advantage that tutorial advice to the students can be based on analogical remindings to former mistakes they may have made.

  2. Conveying normative knowledge. Here, cases are used to tutor normatively correct knowledge. A good example are the ASK systems developed at Northwestern University and Bloch & Farrell's early DECIDER program. Cases are carefully selected and/or constructed in order to demonstrate to the students prototypical correct and incorrect behavior. Usually, presentation of cases is integrated into a more encompassing learning environment that forces the student to become actively involved into a decision making or problem solving activity. This is important for both the CBR component, since the case indexes are tuned towards a specific application context, and for the human student, so that she can mentally index case information adequately.

  3. Capturing and structuring students' problem solving experience. In this instructional scenario, students construct their own cases by keeping more or less structured records of their learning and problem solving experience, records that can be stored into and retrieved from a case memory. This use of CBR techniques is particularly useful to support exploratory styles of learning. For instance, Thomas Schults CABAT program keeps track of the experiments and predictions students conduct in a computer-simulated physics laboratory, indexes them into a case-memory, and retrieves them at appropriate times to remind students of similar experiments and mistakes they have done before. In a similar vein, Reimann, Schult & Bellers SeeChess program allows a student to keep records of what she considers important about example solutions (for chess endgames) and retrieves these cases in later problem solving contexts.

  4. Instructional planning and design. Programs such as pioneered by Kolodner's SCI-ED use case knowledge in order to design instructional settings (for instance, laboratory experiments) and plan instructional content presentation (for instance, an hour in a school class). In this application context, there is no direct interaction with the students and the CBR system: Design and planning is done off-line. Furthermore, the actual interaction with the student is executed by a human teacher, not by the system. CBR programs play more the role of an advisory system here than of an expert planner or teacher. For instance, the CBR system may retrieve from its case library a number of solutions for an instructional planning task, and it will be left to the teacher to select one to develop further.

Issues Raised During the Discussion

One point that became clear was that for instructional purposes, cases will usually need to be carefully assessed, processed and indexed. That is, case representations will not be only the 'raw' data, the episode as it took place in the world, combined with a couple of surface-feature-based indexes. Rather, cases representations need to be selective, partially abstracted, and flexibly indexed. One reason for this is that a case representation in an instructional context needs not only be understood by experts, but by novices with various degrees of general knowledge about the domain, and with various learning styles.

Hence, 'case acquisition' becomes important - and becomes also a potential problem for system development, in analogy to the knowledge acquisition bottleneck in standard expert systems. A related notion is that cases will often be used as examples (both positive and negative ones), hence have to be constructed/selected in order to point out a typical good or bad move. Another issue raised was that developers of case-based instructional systems must be clear about their overall goal: Are students supposed to induce generalizations from cases and form abstractions, or is the intended knowledge structure an enriched mental case memory? For instance, in teaching law to American students, the core knowledge structure students acquire is a mental case library. Little is done to advance their knowledge about the general characteristics of the law. Clearly, in other areas cases are used more in the sense of instances with the expectation that students will induce from cases generalizations and will use case-specific knowledge only to the degree that it adds to generalizations relevant additional information. Making clear to students what is expected from them right at the beginning is important, since human learners are quite adaptive to perceived task demands.

There was also agreement among the participants of the discussion that teaching in general must be more than presenting cases. This is even the case in areas where case-based reasoning is the predominant way of solving problems. Since we want students to acquire case knowledge that can be used flexibly, i.e., transferred to more than merely superficially similar situations, and can be used correctly, i.e., be modified according to new task demands, cases need to be indexed mentally in terms of not only domain and situation specific, but also abstract indexes, indexes that allow for 'far' transfer. In order to index cases in such a manner, one needs a potentially large amount of general background knowledge and domain specific knowledge captured in other than episodic form. Hence, the relationship between case- based reasoning and other reasoning methods must be clearly analyzed and the usefulness of hybrid approaches in instructional settings must be determined.

Both, in order to teach case-based reasoning in a domain and to teach case-based, one needs a detailed understanding of how cases are used by humans in the respective domains. It was mentioned in the discussion that for many areas, in particular for professional fields, we do not have such an understanding yet and that it would be helpful in general if more systematic studies of the actual construction and use of cases in diverse areas would be conducted.

An issue that was raised but time did not allow for deeper discussion was how to deal with cases that result from collaborative problem solving and how to adapt case-based instruction to collaborative teaching scenarios.

Multiple Student Modeling in Group Learning Environments

H. Ulrich Hoppe and Mitsuru Ikeda

GMD-IPSI, Darmstadt, Germany Osaka University, Japan

Mainstream ITS research has recently been criticized for its neglect of supporting human-human interaction and social learning (as opposed to individualized instruction). Given the current state of networking and distributed computing, there are no longer practical or technical reasons for intelligent learning support being limited to the individual case. Though learner modeling for groups of learners (and possibly a teacher) poses new problems, the availability of human support and e.g. peer-to- peer cooperation can also help to avoid existing problems in generating adequate, personally meaningful, feedback to learners. Generally speaking, intelligent subsystems may help in the tasks of knowledge assessment and error diagnosis (Hoppe, H.U. (1994). Deductive error diagnosis and inductive error generalization. Journal of AI in Education, 5 (1), pp. 27-49), whereas the actual tutoring maybe left to human-human interaction of different types. Of course, also the cooperation between humans should be technically facilitated.

First, different group learning situations have to be distinguished and their specific requirements have to be formulated. Examples are teacher-centered classroom situations or unmonitored group learning in which several students have individual access to interactive learning environments. Whereas in the presence of a teacher or human tutor, individual problem solving phases (exercises) may only be monitored and analyzed locally and feedback may be given directly to the teacher, unmonitored situations require an integration of the multiple student models to infer the adequateness of cooperation between certain individuals. Of course this integrated use of multiple student models is more demanding and interesting.

Taking a more theoretical view on multiple student modeling, different types of student models can be analyzed with respect to their "additivity" (i.e. their formal characteristics for integration). This corresponds to the general problem of knowledge fusion from various, potentially overlapping, knowledge sources.

Overlay models are a simple case, since here we can ideally assume complete "additivity" of the individual portions of knowledge. If "buggy versions" are associated to correct rules, things become more complicated but there are still some integration strategies that appear to be tractable. The most challenging variant of the problems arises for student models that are completely synthesized from examples (as e.g. in the FITS/THEMIS framework; Ikeda, M., Kono, Y., and Mizoguchi, R. (1993). Nonmonotonic model inference - A formalization of student modeling. Proceedings of IJCAI '93, Chambery (France). pp. 467-473). Here it is no longer clear which portion of student A's knowledge corresponds to which part of student B's knowledge, and even if such correspondences can be established, the knowledge may still be contradictory. As a first approximation to these problems, the different approaches and their difficulties will be analyzed. However, on this basis we can already formulate research and implementation strategies.

Press here to continue reading more abstracts.