Author: Diane Baird

Royal Melbourne Institute of Technology

Keyword: Multimedia Education Unit, University of Melbourne, Learning Engine, multimedia, World Wide Web, interactive, Computer Aided Learning, CAL, resource sharing, feedback, authoring, teaching system, flexible learning, reflective practice, concept maps, graphs.

Article style and source: Interview. Original ultiBASE publication.

How would you describe 'Learning Engines'?

I see it as a framework within which teachers can conceive and efficiently produce unique learning activities for use across the Web, without having to worry about inventing or re-inventing the delivery methods, protocols and standards themselves. These learning activities are, or should be, richly interactive in the manner of exercising a student's understanding of some particular aspect of a discipline area. This is over and above the usual type of multiple choice question. At the same time, the framework provides the necessary administrative components, such as logging student progress, reporting back to teachers. The aim is to develop a pool of these rich objects that can be shared across the institution.

How did the Learning Engines Project come about? In the last five or six years, people have been working on CAL (Computer Aided Learning) materials. Much of this work has come through an era in which we developed relatively large, self contained modules. We put a lot of effort into producing these materials that we felt would contribute to the education of students. Typical catalysts in this were authoring applications like Hypercard. A lot of time, effort and money went into it. Looking back, it hasn't always been universally successful in changing the way education is delivered. There haven't been the continuing gains that one might expect if the same amount of effort was put into improving teaching practice for example.

Modules tended to be developed in isolation. We embedded content in the software and then it was difficult to extract the content out of the software, and therefore very difficult to apply either the software or content to other areas. For example, in transferring to another discipline area, it's very difficult to just access small parts of that particular software development, or to use the mechanisms of educational interactions behind them. So, developments tended to reside in departments, or even in isolated pockets within departments; they were not shared terribly effectively and were expensive to produce. In addition, there was the problem of getting people to carry out effective project evaluation.

Coming into this Web-delivery model means that suddenly it is possible to deliver fairly small items. I mean, the HTML page or the image or other resource can come from anywhere. It can be dropped into a common environment where we can share bits and pieces. In essence that's where the opportunities start. With Learning Engines we are trying to create the sort of functionality we were trying to put across in the CAL modules, but this time situating it within this Web-based delivery framework where we are able to share it between projects. We've got a reasonable understanding, I think, of how materials might be assembled and put in front of students in this Web-based model, but what I have been trying to do with Learning Engines is in re-developing interactivity at the client-end.

What do you mean by interactivity? People seem to react to that word, and it is probably used rather loosely. To me, it represents a level of discourse between the teaching system and the student, so that there's a sequence of actions and responses. The Learning Engines Project is looking at the richer end of interactivity between a computer system and the student. Of course, at another level the Web offers a whole range of other learning modes that also amount to interactivity (computer-mediated conferencing, list servers, email links between student and tutor and so on). Those are also terribly important.

The type of interactivity, or learning interactions, that went on in my earlier CAL work was very much based on the tutorial style session. We presented students with a problem; students would attempt to solve aspects of the problem, enter a response and were given feedback. They were encouraged to concentrate on the problem. If they got stuck, there was a first level of support from the computer, providing a number of interactions. Then, because the sessions were delivered in the context of a tutorial workshop, there was also a tutor standing by, as a back-up, to handle what was not appropriate or practical for an automated model. Having the human tutor was a good safety valve. I think that helped us develop an understanding of what a computer did well, and it did certain things very well, but also where the limits were, without dropping students into a system that, educationally, could have been a retrograde step.

This approach is different, but is it better from an educational point of view? As soon as you talk about flexible learning or self-study materials (which seems to be where people are currently tending to head) and make these materials available for students any time, any where, I dare say there is a potential advantage. At the very least, there is the potential to handle certain selected aspects of a course in this manner, and to then be able to improve the quality of face-to-face teaching by concentrating on the aspects of the course that cannot be handled in this manner. I guess that this face-to-face contact, whether conducted on-line or live, is really going to be what makes a particular course educationally better. However, I think on-line delivery is going to present us with different problems. Teaching staff at least initially are going to have a much poorer understanding of how students are using their materials. If students don't communicate in some way their problems back to the teacher, we've lost a level of feedback that we have grown accustomed to. I suspect we don't fully appreciate the full value of feedback we get even from something as educationally `unsound' as a large group lecture. But, at the same time, we have opportunities on the Web to gather some of that back.

The Learning Engines framework has two sides. The first side is in getting the material in front of students, in an authoring sense, and defining how these operate from the students' point of view. But the side that I think has been greatly under-played, both in the CAL materials, and increasingly, in this more uncharted area of Web delivery, is in feedback to the teaching staff. Feedback, so that teaching staff can immediately understand where students are having problems or how they are actually using materials. Teachers need to understand who is using them and how effective they are in reaching the students who've got the problems. That's very important.

The area that I think desperately needs practical products and research is in facilitating reflective teaching practice. That is an important aspect of the Learning Engines Project. As well as the authoring and production of materials, you have the complimentary ability of being able to efficiently examine students' actual usage of those materials and use the feedback to improve both the materials and class teaching.

What we're moving into is a degree of centralisation on the Web. That has certain strategic advantages, as well as the disadvantages of centralised control, but one aspect we can exploit is that we can monitor different patterns of activity literally as they occur. One of my areas of research is looking at students' usage of materials and mapping those into a reporting format that can be understood and used in practice by teaching staff.

So you can track individuals to find out where they're going and how they are using the materials, but not necessarily what they are learning.

That's right. It is possible to 'map' an individual, to generate reports of the answers that all students are giving, but you need to be wary of linking this directly to evidence of learning. I have seen too many examples of false interpretations of these type of data to take anything too literally. It does however provide a potentially very useful tool. For a particular question, a list of all the answers and the frequency of the answers can be produced. By quickly scanning that question, a teacher can say, "Well, these appear to be the common incorrect responses...". You can get a pattern of the sorts of mistakes that are being made. It all comes back to your experience and strength as a teacher; your understanding of the material you are putting across and the way students approach it. If you can say..."students appear to be demonstrating a particular misconception", then you can go on to the authoring side and either edit that particular component to clarify the point, or give students an optional component to assist them, or bring up the issue in the next live tutorial.

Why do the computer programs make the teaching/learning process better?

If you have a one-to-one situation and if you've got a good teacher, I dare say that's the best kind of interaction. However, you may have instead classroom tutorials of, say, sixty people and little opportunity to give feedback to the individuals in the form of corrected exercises. The computer, in this particular instance, has the advantage of providing the first level of interaction, freeing the tutor to go around and service the individual needs of the students in a more efficient way. That has been a situation where I have seen computers really work effectively. Whether or not it is the most cost effective way of making the process better is another question.

You have said that it will be amazing if new technology does not lead to dramatic improvements in teaching and learning. Why?

Did I really say that? It is also going to lead us into all sorts of traps as well. In the end, you are going to get what you pay for, one way or another. There are easy ways of changing technology that really aren't going to change teaching and learning a great deal. In fact, they may make things worse. At the moment, with a move to the Web, people are rushing to put their notes on the Web for example. For some reason, they seem to think students won't want printed notes. I can't imagine many students who won't want a set of printed notes. One way or the other, they print them off themselves. It's a ridiculous waste of effort. You really have to question the validity of the activity. There may be other reasons for putting the notes up, but if they are going to be printed off, not much has been achieved. One other factor I keep coming across is that people don't seem to realise that materials put up on the Web in learning terms may be significantly inferior to traditional materials in many ways. Students are actually reacting against that.

That's the reformatting trap that so many people are falling into. How do you help them move from reformatting to a reforming of approaches to education?

I think this is where seminars and a unit like this [Multimedia Education Unit] which is charged with academic development, can bring awareness of issues to the fore. I think issues are being discussed much more openly than a year or so ago. Also, for quite some time at Melbourne, it's been my impression that we've had more facilities than good materials to put on the Web. I hope that methods such as Learning Engines can help people develop 'quality' materials for their students. Of course, we will need a greatly increased level of access in order to disseminate ideas and materials effectively in the future.

Using the Web requires different sorts of skills, both in writing for the Web and in using it effectively. One of the issues for teachers is how to organise information so students can use it effectively.

I think that's what Learning Engines is trying to address. It is going beyond software delivery system solutions to methods of customising activities. The question is, how can we exploit the opportunities the computer can give us in order to set up an activity where a student can explore, or be guided through, or interact, or create an understanding effectively? But, we need to do that within the context of the broader delivery system. We don't want to be re-creating...solutions. We want to be able to take solutions that one department has developed and use those. The types of administrative functions...need to be developed once...so we're not replicating all the routine types of requirements of a piece of software. That's where the Learning Engines project comes in.

So it is in part a delivery approach. What about the educational part of the equation?

What drives that part of the equation are the academic needs. In the end, it is essentially driven by the perceptions and the understandings of the academic who has a teaching problem. In looking for solutions, we're providing a way for them to focus their attention, as much as possible, on the educational functions behind solving it, rather than on the technical delivery and software development aspects. Coming from that angle they are better able to apply learning activities that may already exist or be able to create designs for new ones.

So in working with `Dr. Staff', might it be that the solution of the teaching problems has nothing to do with using computers? Certainly. I think in our position here, we always have to draw attention to the fact that the computer-based solution is only one tool. It's one opportunity you have. All too often however it appears that people are saying, "Here's a great solution, how can I apply that to my teaching?" That's precisely the wrong way around.

You stand back, away from computers, and say, "we've got a problem with students appearing not to understanding certain relationships, or concepts, how can we solve it ?" The computer may provide that solution. In science, information is expressed in the form of graphs, but they very seldom feature in any sort of teaching/assessment activity. They are difficult to mark because they require very careful personal attention that tutors can't afford. But here's a situation where a fairly small piece of software can allow teachers to ask students to express themselves graphically, in a manner consistent with the scientific representation and with the computer able to interpret, to a reasonable degree, their efforts and provide a level of feedback.

Once you've identified, even without thinking about software, a way of providing an open-entry format for the particular type of knowledge you want to deal with, it can often become independent of the specific discipline area. It can then be applied to Economics or Physics or Maths or Biology to solve particular problems in those disciplines.

The problem with pieces of paper usually comes back to who's going to look at the pieces of paper. If you say you want students to express themselves and use a concept map to come to their own understanding, well, that's fair enough. But unless you see that piece of paper and include it to some extent in your assessment, you'll have very little understanding of whether students are learning or not. If you have the time and the effort and you can engage with students on a personal level, that's undoubtedly what we'd like to do. But, unfortunately,we don't have time for that. The concept map has interesting possibilities when it comes to thinking up creative activities for students.

Concept maps have been used extensively in education. For that very reason, you might have sceptics saying to you, `Demonstrate to me why it is better to use a computer program for concept maps.' I think you can only take it on a case-by-case basis. At the bottom, it depends on the skill of the teacher who is applying it. If the teacher doesn't have the skill, or commitment to computer based solutions, having a concept map mechanism is not going to make educational delivery significantly better. It depends on the quality of the question that is behind it. I think that's what we're trying to do with Learning Engines: to raise the level of skill, enthusiasm and commitment of the teaching staff.

The difficulty is how to design a learning interaction, given the range of possible learning interactions that can be applied. It is as much a content problem as it is a technical one. The ideal, I suppose, is to harness the effort of the academic staff, in thinking about educational and content-related problems, to implementations where the technology is as transparent as possible.

How would you define excellence in teaching and learning? I suppose there are many angles to excellence. Looking at it in terms of the technological solutions, here's an example. A group, or a couple of students, in front of a computer debating an issue or a problem; working for an hour or so in a tutorial; entirely focused on the learning task, and actively engaged in discussion between themselves and bringing in a tutor. When we were first putting materials in front of students, to see that used to send shivers up my spine.

I think the other issue that is important relates to the reflective practice of teaching staff. I know excellence is not just putting up materials and letting them run by themselves. But where the system can give the opportunity to teachers, if they are good teachers, to observe how their materials are working and then to respond, to improve the learning experience for their students and at the same time improve their own understanding of teaching and learning, that approaches excellence.

Materials/Activities will never be perfect first up...It's part of an iterative developmental cycle. The information flow about what the student does, their feedback to the teacher, and how teachers in turn respond back to students, either improving or adapting materials, is critical. The shorter that iterative cycle and the more it becomes part of a teacher's way of thinking, the more some level of excellence is achieved. I genuinely think the Web has potential in that regard.

Why?

The components can be smaller. They can be focused more carefully on specific learning problems, and they can be integrated with other activities very easily. The Web also provides the communication channels for this two way transfer of information.

What are the next steps for the Learning Engines Project?

It's difficult to talk definitively about it, because as yet, it still hasn't fully evolved. It will only emerge from the ideas that are coming to the fore in a number of projects that we're currently working on. One of the exciting things about the Learning Engines Project is that it is actually driven by and evolving out of the projects themselves. What I can say is that a core set of components that make up the framework is at proof of concept stage. I have been demonstrating these around the university and found a deal of interest although I make the point that what we have is not a product as such, it is a concept that will develop into a practical framework of interconnecting components. I am advising people that if they are interested to discuss their needs and ideas with me in the first instance, that will help us both in understanding the issues in different disciplines. We are working now with a number of departments in prototyping customised activities: Veterinary Science, Chinese, Psychology, Chemistry and others. At the same time we are investigating incorporating activities into commercial course delivery systems. The real target date is at the start of next year when these first projects are put to students.

Interview recorded on 26 May 1997.

For further information about the Learning Engines Project, contact Paul Fritze.

About the author