Computer Aided Assessments for formative assessments

With the ever-increasing number of students entering Higher Education (HE) there is a pressing need to use alternative assessment methods which are as, or indeed more, effective than paper-based assessments. This increase in student numbers has made the process of providing students with formative feedback much more difficult. There are many methods that can be implemented into a course to provide formative assessments such as peer-assessments, class-tests, self-assessments, etc. This section will look into the use of Computer-Aided Assessments (CAA) which is a tool that has been used in HE for a considerable time.		What are Computer-Aided Assessments?
		CAA package used at Brunel University
		How CAA can be incorportated into courses and modules?

What are Computer-Aided Assessments?

Definition

Computer-aided assessment (CAA) is just one of the many terms used to label the use of computers to assess student learning (Brown et al., 1997). CAA software is used to deliver, mark and analyse examinations and record, analyse and report on student achievements. An extensive overview of CAA is provided by Bull and McKenna (2004).

Advantages and issues of using CAA

There are many advantages of using CAA, some of which are:

The flexibility to use CAA for different types of assessments, including diagnostic, formative and summative.
The use of different question types within assessments, which ranges from the standard multiple-choice to hotspot type questions.
Flexibility of allowing students reattempt of questions and assessments.
Providing instantaneous feedback.
Saving marking time for academic staff.
Time efficient for assessing large groups

There are, however, still some issues with using online assessments. The initial time and skill required to develop 'good' assessment questions can be substantial. Expertise may need to be brought in to help with the initial development of a question database. Another difficulty is the marking of open-ended questions. Many open-ended questions still need to be marked by a human rather than a computer system. More discussions on the advantages and issues of using CAA can be found on the JISC website: http://www.jiscinfonet.ac.uk/InfoKits/effective-use-of-VLEs/e-assessment/assess-ads-disads

The advantages of CAA have been widely stated and discussed in the literature for a number of years now. Many case studies have been discussed where qualitative data has been collected in terms of student and staff perceptions of CAA. Although this is important and useful information it does little to inform practitioners of how to best make use of the technology available or quantify the benefits students can achieve on successful engagement with CAA. Over the last five years research has been undertaken at Brunel University to do just this. Not only have we collected evidence to show that on successful engagement with CAA students can improve performance in paper-based tests/assessments (Gill and Greenhow, 2008) but we have also developed objective questions that can assess a wide range of skills which differ in difficulty and which can test higher-level cognitive skills (Baruah, 2007).

CAA package used at Brunel University

At Brunel University, the CAA package Questionmark Perception is used to develop questions in the area of mathematics. The database that has been developed over the last 10 years is known as Mathletics and has been developed by Dr Martin Greenhow (http://people.brunel.ac.uk/~mastmmg/). The current database contains approximately 1800 question styles, each contains random parameters which results in thousands of realisations of one question style (Greenhow, 2008). The questions cover lower level mathematics i.e. range from GCSE to undergraduate level 1 mathematics. Extensive work has been undertaken by Martin Greenhow and his team to provide questions with extensive feedback which makes the questions (and assessments) a valuable learning resource to students (rather than just an assessment tool).

How CAA can be incorporated into modules and courses

CAA is widely used across the university in different ways. Below is a brief overview of how the Mathletics database has been used to integrate assessments for diagnostic and formative purposes into modules.

Diagnostically

Diagnostic tests have been set up for all level 1 mathematics students and Foundation of Information Technology students. Students have to complete the tests by the end of their first week at university. Personal tutors receive the results of the assessments and then feed back to students areas of weaknesses and improvements that need to be made. In some cases students are asked to repeat the assessments. The results from the diagnostic test are also used to change initial course material to bridge gaps between students’ level of understanding and expected level of understanding. Diagnostic tests have been used in this way successfully for the last 10 years and have proved to be very efficient.

Formatively

One of the key features of the questions developed as part of the Mathletics database is the extensive feedback provided. Students are told, where possible, the mistake (or mistakes) that they made, given the fully worked solution and are provided with some generic feedback. A recent addition to this feedback was a 'related material' functionality which pointed students to other resources that they may find helpful (such as links to the national mathcentre). Research undertaken at Brunel University found that once students engage with the assessments it benefits their learning of the subject area (Gill and Greenhow, 2008). However, getting students to engage and undertake formative assessment tasks is more challenging than just making assessments available. A method that has been used at Brunel to encourage students to engage with formative assessment tasks is to incorporate computer lab sessions into students’ timetables. By having an academic member of staff or a post-graduate student present, students are more willing to engage and interact with the assessment system knowing that help is on hand if needed. The presence of an academic member of staff has proved successful with level 1 mathematics modules. Students attended weekly lab sessions and engaged and interacted with the assessments even though no marks achieved from these assessments contributed to their final module grade.

References

Baruah, N. (2007) Computer-Aided Assessment of Higher-Level Calculus: Design and Evaluation. Published Ph.D thesis, Brunel University.

Brown, G., Bull, J. and Pendlebury, M. (1997) Assessing student learning in higher education. London: Routledge

Bull, J. and McKenna, C. (2004) Blueprint for Computer-Assisted Assessment. London: RoutledgeFalmer

Gill, M. and Greenhow, M. (2008) ' How effective is feedback in Computer-Assisted Assessment. Learning', Media and Technology, 33(3) pp. 207-220.

Greenhow, M. (2008) ' Mathletics - a suite of computer-assisted assessments', MSOR Connections, 8(3).

Greenhow, M. (2010) ' Development of Computer-Aided Assessment of Mathematics for the First Year Economic Students'. In Responding to Mathematics Problem: The Implementation of Institutional Support Mechanisms, Maths, Stats & OR Network, 64-69, ISBN 978-0-9555914-6-4.