u!magine

eExam Trials

Description

The project explored the suitability of current technology that would allow a massive change in assessment processes and allow students to take their exams online. For CSU and many universities around the world exams are a very real part of the student experience. They are also an important of assessing student for a variety of reasons - they allow the university to verify the students identity and match them to the work they present, the test the capabilities of the students, they provide a relatively simple way of assessing large groups of students and are a key component of many professions formal accreditation.

Allowing students to take exams online would provide students, especially those study away from a campus location, a much easier way to complete there exams. The process could provide students with greater flexibility around when and where they do their exams. There are also significant benefits for the institution especially in areas where it may be difficult to provision a paper exam such as rural and remote students and those studying overseas.

Aims

The aim of this trial were:

  1. Evaluate the use of online proctoring services to replicate current exam conditions.
  2. Evaluate the viability of these services from a technical aspect and highlight any problem areas or considerations that need to be made.
  3. Evaluate the use of Blackboard to provision eExams for students and replicate common exam questions.
  4. Evaluate the perceived improvements for the Exams Centre and the effect on the Student’s Experience that eExams may have.
  5. Explore the potential and possible application of these services and their  applicability to broader concepts of online assessment.

Approach

The project trialled two invigilation or proctoring services that utilise online tools to provide supervision for students taking exams. Both vendors had their practices vetted by the CSU legal team to ensure that student data and privacy were protected during the process. To keep things simple and reduce the number of moving parts and possible failure points, the exams were limited to multiple choice, short and long answer questions and were recreated as Blackboard Tests. Blackboard Tests provided the necessary security measures and had the ability to write straight to the grade book which helped to remove the double handling of marks and grades. The first run on the system and process was for a midterm exam which was low risk to students and had a fall back provisioned. Students across 4 additional subjects were offered the choice of completing an eExams to replace their final exam. eExams were run across the 24 hour period before the paper based exam. This allowed students flexibility in what time they could enrol and allowed the paper based exam to be a fallback if there was a catastrophic failure of the technologies involved. Students were provided with instructions on how to test both their computer and network connection to ensure they would be able to connect to the system and complete the exam. The exams were run without significant problems for the students, but there were some administrative issues with one of the vendors. The second phase of the trials continued with just the one vendor and additional subjects.  

Outcomes

A number of recommendations were developed based on the outcomes of these trials:

  1. The need to establish a “business as usual” process between faculty, the exam centre and other required staff.
  2. The goal to move to a stage where all students can be offered “Online” as an exam centre option in order to to take part in an eExam. This opt-in approach would be offer the safest and most conservative route to adoption, but it also provide time to develop capacity within CSU to work in this way and students to transition to more of their learning experience being online.
  3. A need to Identify responsibility for creation, facilitation and support of exams.
  4. Test other technology options to provision exams which may vary the type of exam questions that student can answer and how - e.g. drawing or sketching diagrams for an answer. Blackboard does have a stable and easy to use environment, but these trials did not explore other options and so cannot say definitively that Blackboard is the best, or even the most suitable solution.
  5. Further testing of the invigilation services on more extreme edge cases of connection options for our students such as mobile and satellite connections. For CSU’s vast spread of students it is those on the edges of technology that are the most vulnerable and difficult to service. Find ways that would enable them to participate more in online learning, and reduce their need for travel could be a significant cost saving as well as greatly improve the learning experience.  
  6. Explore the use of invigilation technology to support disability students and increase equity and access. The use of digital technologies in exam situations have been limited due to the exploitation that they could enable. A system like this which includes invigilation and monitoring of what is on screen reduce that risk significantly. There may also be significant benefits for students with disabilities being able to use a computer during an exam and may reduce the need for other support and assistive technologies currently used.
  7. Explore the use of the invigilation technology to conduct other assessment tasks. The recording of video and audio could open up the service to other types of assessment other than examinations. Students could be videoed performing tasks and demonstrating skills and the invigilation environment could allow that to happen in a verified and online platform. At a meeting with ProctorU staff they also described a future service that would allow students to simply log in and use the platform to record themselves completing a task, like a test or exam,  without an invigilator. University staff would instead be charged with verifying the student and if they completed the task properly and not pay for the invigilator.