[Home]
 
[Current Issue] [About Us] [Subscribe] [Search] [Events] [Resources]
 

The development of self- and peer-assessment strategies for a design and project-based curriculum

Author: Judy Hodgman

University of Tasmania

Keywords: Assessment, University of Tasmania, design, project, teaching, learning, values, peer-assessment, curriculum, evaluation, VALUED.

Article style and source:Moderated. Original ultiBASE publication.

Contents

Introduction

The video and teachers' learning guide 'The VALUED Approach to Assessing Design Project work' has been developed as a resource for all teachers and learners involved in a design and project-based curriculum. The acronym 'VALUED' is used to describe the integration of a design process with a criterion-based assessment process. It provides a teaching and learning strategy for the implementation of self- and peer-assessments. The letters mean:
  • Valued
  • Assessments
  • Link (students)
  • Understanding of
  • Evaluations and
  • Design.
Through the video learners can observe five phases of a design project which was undertaken by third year design and technology, trainee teachers. Each phase demonstrates how students use and develop criteria for design project work and how their chosen criteria are used for self- and peer-assessment purposes. The task chosen to facilitate the process was a graphic design project, but other projects have since been tested and support the use of the VALUED process. Each chapter of the book contains a brief overview of the phases of the VALUED process.

Background to the project

Recent research, undertaken by the author (Hodgman, 1994), suggests that the design process and the objective-based assessment process have a dramatic effect on the way students approach their design project-work. This is because the two processes were found, at times, to be in conflict with one another. The research investigation raised important questions:
  • How much of the assessment process relies on the teacher's ability to plan and execute an educational experience?
  • What part of the assessment process recognises students' understanding of their own learning?
A second stage of research sought students' and teachers' responses to using both a design process and an objective-based assessment process (during design project work). This investigation provided enough data for statistical analysis to occur. Findings included the identification of the following factors:
  1. the assessment process (internal),
  2. the design process,
  3. the assessment mechanism (as a limitation),
  4. external assessment,
  5. formative design evaluations,
  6. summative design evaluations, and
  7. product outcomes.
Points three and four describe the mechanism for assessing performance and content. Points five, six and seven describe elements in a design process and point 1 suggests that a cause /effect relationship may be occurring. However, only factors 3, 4, 5, 6 and 7 were shown to be consistently reliable. Further, these factors were also shown to be relatively independent. The independent characteristics of these factors were an unexpected outcome, particularly as students used both a design process and an assessment process to guide their design project work. However, further analysis of these data found statistically significant correlations between:
  1. students' use of a design process,
  2. students' development of formative design evaluation skills,
  3. students' development of summative design evaluation skills, and
  4. design outcomes (product).
The research suggests that the assessment process offered teachers the opportunity to evaluate their own performance (designing and implementing an educational experience), as students' results depended on this performance. This was because nationally agreed assessment criteria were interpreted and assessed (by the teachers) and were used, in turn, to facilitate learning. Less evident in the assessment results were students' interpretations of the assessment criteria. The investigation supports the conclusions that:
  • The design process is beneficial in developing design evaluation skills and design evaluation criteria.
  • The objective-based assessment process is independent of the design process.
  • No relationship exists between the development of formative and summative design evaluation skills and the objective-based assessment process.
  • The objective-based assessment process is teacher driven and accommodates teacher values.
  • Teacher interpretations of learning outcomes support teachers' values.
  • The design process is student-centred and supports students' values.

The assessment debate

Whatever strategies were used to encourage deeper learning, students would always be powerfully influenced by the assessment system they were required to work to and this, according to Beaumont (1994) was a critical activity:
one that is often shrouded in mystique, governed by tradition, and has the tendency to be notoriously inadequate (p. 1).
McGaw (1986) argued that assessment needed to be better understood particularly with regard to the way in which students represented problems and structured knowledge. This led to the following questions:
  • What are the criteria for a reliable measuring instrument?
  • What measuring instruments are currently available that have been validated?
  • To what outcomes can these instruments be applied reasonably and informatively and that can be used across different domains?
  • Should schools set their different objectives or outcomes?
  • Are students disadvantaged if the measurement of outcomes and the assessment applied to them are not clearly defined and to some extent standardised?
  • Is it possible to provide a clear picture of students' abilities when they leave school in terms of grades?
  • Are students disadvantaged if the criteria chosen to assess a specific task are too controlled by the teacher?
  • Are the design educational experiences consistent and reliable enough to measure the skills or attainment they set out to assess?
  • Are assessment procedures advantageous to the child's development?
As educators were generally shown not to have this information, it should be recognised that they are unable to provide reliable feedback either for the students or for future curriculum design. This information, for Schafer (1991), was essential as teachers in schools spent more than one-third to one-half of their instructional time dealing with assessment and assessment related matters. Validity in criterion-referenced assessment was taken to mean content validity and included the following three elements:
  • the domain of adequately defining the criteria,
  • appropriate and applicable content design, and
  • answering the question: are teachers assessing what they think they are assessing?
Crooks (1988) warned, however, that evaluating students on criterion-referenced evaluations could encourage competitive situations, as making comparisons, particularly product comparisons, was shown to be part of this assessment process. He pointed out that teachers using objective-based assessment criteria used assessments for norm-referencing, which suggested that reliability was guaranteed within the consistency of decision making by the teacher. Inter-marker reliability, therefore, was seen as being essential if equity and equality in assessment is to be achieved.

The discussion on the issues surrounding assessment is summarised as follows:

  • students are expected to guess the nature of assessment requirements,
  • assessment requirements are often a separate situation to what happens in lectures and tutorial sessions,
  • assessment techniques and procedures are used to cover only a small range of objectives,
  • the focus of assessment is often unclear to the students,
  • practical assessment is time consuming, and
  • assessment is usually used for summative rather than formative purposes.
Although these types of assessment propositions were not always representative of all assessment contexts, Beaumont (1994) suggested that meaningful assessment should reflect the nature of the profession, vocation or practice under scrutiny and at the same time reflect the true nature of the discipline.

A rationale for the teaching and learning model

The author recognised that a teaching and learning model was needed that integrated both a design and assessment process. It is argued that an evaluative model that supports students' interpretation of learning outcomes as well as the teacher's, offers opportunities for students to experiment, unencumbered by teacher limitations (expectations). Further, in a student-centred teaching and learning environment, too much teacher intervention was shown to destroy the meaning of 'student centred' as this limited the development of formative and summative design evaluation skills.

Many educational theorists including Crocker and Cheeseman (1988a and 1988b), Gibbs (1990) , Gipps (1994), Haertel 1993), Kimbell (1982), Linn (1991), Resnick (1989), Williams (1996), concluded that the teacher's assessments restricted students taking responsibility for their own learning. They found that the main separating factor between student- and teacher-centred learning was a lack of balance in assessment. Further, self- and peer-assessment strategies were an appropriate alternative for assessing the design and project-based curriculum. They agreed that access to metacognitive processes came from negotiated self-assessment where pupils gained awareness of their own learning strategies in the light of educational objectives. Implicit in this was the need for students to self monitor, reflect and set their own goals. These conclusion accord well with the theory that all attempts to develop criterion-referenced assessments in the design and project-based curriculum had met with problems for both teachers and students. Particularly as educational measurements could not yield reliable, consistent, and replicable results.

The teaching and learning model, presented in a video, supports the theory that self- and peer-assessment strategies are needed to develop design evaluation skills. This is seen as a significant paradigm shift, but welcomed by the theorists who linked assessment results with self-esteem, and learning behaviour with achievement motivation. The model supports a student-centred approach that focuses on the development of summative and formative design evaluation skills. In this approach both process and outcomes are evaluated by the students. This provides a framework that links:

  • a) a design process (where design evaluation criteria are developed), and
  • b) an assessment process (where assessment criteria are developed as a result of the student).
Each student's chosen criteria are then used for self- and peer-assessment purposes.

Involving students in the design of assessment criteria enables them to focus on the design problem and the use of a design process. A typical design process may include some or all of the following activities:

  • the recognition of a problem,
  • design restrictions,
  • research and investigation,
  • design development,
  • design decision-making,
  • product evaluation,
  • prototyping, and testing.
It is advised, however, that a cyclical design process is beneficial as it offers opportunities for students to consider specific design outcomes in advance. 'Testing' existing products offers the opportunity to make many design evaluations. This is also considered advantageous to helping students recognise the environmental, social and economic implications of their decision-making choices. Further, changes to ideas and values are shown to be made easier by this approach.

A review of the literature also suggested that individual values are important to the decision-making process and, as such, should form part of any teaching and learning model. This was taken into account and the following model was developed based on the need to provide students with:

  • a collaborative, interactive teaching and learning environment where design evaluation criteria (for assessment purposes) are developed as a result of the student and not as a result of the teacher, and
  • an opportunity to gain an understanding of their own learning.

The VALUED process

The VALUED process describes an integrated process that links students' values, with assessments, design evaluations and a design process. The acronym means:

Valued
Assessments
Link (students')
Understanding of
Evaluations and
Design.

The objectives of the VALUED process are to encourage students to:
  1. interpret and discuss the design problem, the product's value (in terms of real need in the market place) and personal values,
  2. develop design evaluation/assessment criteria for use in a design process,
  3. make links between a design process and an assessment process through an understanding of design evaluation criteria, the use of a design process and product outcomes,
  4. critically evaluate the criteria chosen by each individual student,
  5. peer- and self-assess design outcomes based on the student's chosen criteria (for assessment and design purposes).

The video

The video addresses the above five phases:

PHASE ONE: Values (Design Brief)

During this phase the design problem is outlined by the teacher and discussed by all students. Students' recognition of their own values (as they apply to the design problem), as well as those of others, are evidenced. Collaborative, interactive discussions and 'brain storming' exercises are used to facilitate the exchange of ideas and values. Students are encouraged to share the design problem and their personal values with the teacher and other students. This is done in a non-threatening teaching and learning environment where all ideas are accepted and where more ideas are preferred to less. The objective is to encourage students to interpret and discuss the design problem in terms of the product's 'value' (social, environmental and economic), and the students' personal values as they apply to the problem.

PHASE TWO: Assessment and Design Criteria Developed

Students are encouraged to make lists of design evaluation criteria for use as assessment criteria. They are encouraged to discuss their design problem with the teacher and fellow students. The use of formative and summative design evaluations are encouraged throughout this phase. The objective is to encourage students to develop design evaluation criteria for use in their design process. These lists of criteria are needed for self- and peer-assessment purposes (Phase five) and for design development to take place. The use of formative and summative design evaluations are encouraged throughout this phase.

This process is in accord with Miller et al (1978) who suggested that making design evaluations meant applying behaviour. The student selects the desired outcome (evaluation), summarises the situation and predicts the effects of alternative changes he or she might make. The student then evaluates the alternative procedures that would predict the desired outcome, based on such criteria as he or she deems appropriate to the context of the project.

PHASE THREE: Links Made between the Design and Assessment Processes

In the video students demonstrate how these processes may be linked. During this phase, students are encouraged to use a design process that supports their project work and personal values. They are encouraged to develop a list of design evaluation criteria needed to further the design process. They are also encouraged to make links between these evaluations and those needed to assess the project. Their understanding of both processes is an important learning outcome for this phase.

This outcome agrees with Miller et al (1992) who suggested that teaching in Design and Technology has many parallels in terms of providing networks of information that interconnect. This map is almost impossibe to transfer to the student in terms of linear knowledge.

PHASE FOUR: Evaluations of Design and/or Assessment Criteria for Self and Peer-assessment Purposes

Students finalise their design project work during this phase. The main objective is to insure that students gain a full understanding of the design/assessment criteria chosen by each student for their respective design projects. This is achieved through formal or informal discussions and is done in readiness for self- and peer-assessments to occur in Phase five. Students' assessment criteria are presented to the teacher in written form during this phase. This agrees with Gipps (1994), who warned that assessment innovations had been adopted even though they had high levels of uncertainty. The assumption was that they would raise educational standards and control curriculum.

PHASE FIVE: Design Solutions Assessed

By this stage students should have reached a solution to the design problem in readiness for self- and peer-assessments to occur. They are encouraged to participate and accept their role as 'assessors and to use the written criteria (provided by each student) to assess each project. The main objective for this phase is to encourage students to peer- and self-assess design outcomes, based on individual students' design evaluation (assessment) criteria.

Summary

The teaching and learning material helped to overcome many of the assessment problems I faced when attempting to teach trainee teachers how to use and develop assessment criteria for design and teaching purposes. This was because the VALUED model encouraged the development of teacher educators' understanding of instructional objectives and how these related to design project work. Further, it prevented a conflict of interest occurring between a criterion-based assessment process and a design process as the opportunity for choosing between these two processes was eliminated by its use. By integrating peer- and self-assessment strategies with a design process, the resulting developments and use of design evaluation criteria were evidenced.

The teaching and learning material has been used in support of University of Tasmania teaching and learning (Bachelor of Education, Design and Technology) in the following units:

  • Technology curriculum 1 and 2,
  • Technology in the Primary School
  • Design and Technology 1a
  • Design and Technology 2b
  • Technology 6
  • Technology 3 and 4.
Positive feedback was provided by students involved in monitoring and trialing the teaching strategies. The students suggested that:
  • a) the 'VALUED' model, supported a student-centred approach that allowed them to focus on the design problem, and
  • b) as process and outcomes were evaluated by the students, a framework for linking
    • (a) a design process (where design evaluation criteria are developed), and
    • (b) an assessment process (where assessment criteria are developed as a result of the student) was achieved.
Evaluation of the teaching and learning material was also sought from University lecturing staff involved in a design and project-based curriculum (Engineering, Architecture, Graphic Communication and Design and Technology in Education). They agreed that the students trialing the material achieved a high standard of work and projected a positive and professional approach to their design project work. It was generally agreed that the design and engineering professions would benefit from teaching strategies that develop self- and peer-design evaluation skills, particularly as the economic and ecologic advantages of using quality assurance programs increase. I therefore anticipate that the theoretical framework will have a significantly positive effect on the design and project-based curriculum.

References

Beaumont-Kings, C. 1994, 'The Impact of Assessment on Learning', paper presented at AARE conference, Newcastle, November.

Crocker and Cheeseman, R. G. 1988, 'Infant teachers have a major impact on children's self-awareness', Children and Society, 2.

Crooks, T.J. 1988, 'The impact of classroom evaluation practices on students', Review of Educational Research, 58, p.4.

Gibbs, G. 1992, Improving the quality of student learning, Bristol: Technical and Educational Services, p.167.

Gipps, C. V. 1994, Beyond Testing, The Farmer Press, London.

Haertel, E. 1993, 'Evolving conceptions of the generalisability of performance assessments', paper presented at the AERA conference, Atlanta, April.

Hodgman, J., G. 1995, 'An evaluation of the effects of the criterion-based assessment process on design outcomes and student performance', Design and Education , vol. 5, August, pp.3-10.

Kimbell, R. 1982, Design Education, Routledge and Keegan, London.

Linn, R. L., et al. 1991, 'Complex Performance based Assessment. Expectation and Validation Criteria', Educational Researcher, 20 (8) pp.15-21.

McGaw, B. 1986, 'Turning novices into experts', paper resented to the Tasmanian Institute of Research, ACER, Hobart.

Miller, G. G., Williams, R.G., and Haladyna, T. M. 1978, Beyond Facts: Objective Ways To Measure Thinking, Educational Technology Publications, New Jersey.

Resnick, L. 1989, Knowing Learning and Instruction, Lawrence Erlbaum Associates, New Jersey.

Schafer, W. D. 1991, 'Essential assessment skills in professional education of teachers', Educational Measurement Issues and Practice, 10 (1) pp. 3-12

Williams, L. 1996, 'Large Classes, Computers and Assessment', ( Jo Arnold Editor) ,Tertiary Education News, University of Queensland, vol. 6, 1, April, p. 4.

Self-and Peer-Assessment Strategies for a Design and Project-Based Curriculum, Video and Teachers' Learning Guide, FHS DURATION: 25 MINUTES; $AUD 45.

Order from: Judy Hodgman, School of Education, University of Tasmania P.O Box 1214 Launceston, 7250 Tasmania Telephone 03 - 63243298; Facsimile 03 - 63243048

About the author

Judy Hodgmen
School of Education
University of Tasmania
PO Box 1214
Launceston
Tasmania 7250
Email: Judy.Hodgman@utas.edu.au
Copyright © Judy Hodgman, 1997. For uses other than personal research or study, as permitted under the Copyright Laws of your country, permission must be negotiated with the author. Any further publication permitted by the author must include full acknowledgement of first publication in ultiBASE (http://ultibase.rmit.edu.au). Please contact the Editor of ultiBASE for assistance with acknowledgement of subsequent publication.
[up]
Send feedback to manager@ultibase.rmit.edu.au
Copyright © 2001 Faculty of Education Language and Community Services
Document URL: http://ultibase.rmit.edu.au/Articles/dec97/hodgm1.htm
Last Updated: 02-July-1997 by Marita Mueller		 [RMIT University]
 
current II subscribe II about II search II events II resources