Online Learning Model Pilots

As part of the CSU Curriculum Learning and Teaching Sub Plan uImagine worked with learning and teaching leaders within faculties to identify subjects to pilot the elements of the model in the 201560 and 201630 sessions. The goals of the pilots  was to evaluate and refine the model, derive design and implementation guidelines, provide an evidence base to support at scale implementation, and develop an online showcase to underpin future professional development.

Educational Designers Gerard Bourke, Bec Acheson, Kerri Hicks, Stewart McKinney and William Adlong, under the leadership of Lindy Croft-Piggin, have been assigned to each of the elements of the model to provide support to subject teams and to provide scholarly and practical leadership throughout the university in relation to their allocated elements of the model.

The work that was carried out has been used to refine the Online Learning Model as well as develop the Online Learning Exchange. The work and the lessons learnt in the pilots were fed into the continued work to Scale Up the implementation of Online Learning Model. This is a significant strategic initiative that uImagine led with the aim of ensuring that CSU students are provided with the highest quality online learning experiences.

The CSU Online Learning Model (OLM) elaborates seven elements known to increase student engagement. Its goal is to target increased retention, satisfaction with teaching quality, and ultimately student enrolments. In 2015, the faculties of CSU (four at that time) selected seven subjects each to target the enhancement of one of the elements of the OLM. These 28 subjects were supported by a team of OLM Element specialists, the Media Resources Unit and the faculty Education Designers. With time release and skill support, teams of educators prepared subjects for presentation and evaluation in 2016.

Following an initial evaluation, the model was then revised (to Version 2) prior to scaling it up into eight larger courses across CSU during Session 2 of 2016 and into 2017 under the leadership of three faculty-based Quality Learning and Teaching Online (QLTO) leaders. This process has been further evaluated towards the development of the latest version of the OLM (Version 3).

The Pilot Evaluation Process

The focus of the pilot project was on the individual elements of the model, rather than implementing the entire model or multiple elements of the model, within each subject. Survey and interviews with students and staff were the primary methods of data collection. The student survey (referred to as the OLM survey) comprised items related to learner engagement, teaching quality and overall satisfaction from the national Student Experience Survey (nSES) in addition to items curated specifically for the OLM pilot evaluation.

In the OLM survey, students were asked to identify the presence of features of the OLM elements along with nSES questions. As demonstrated in the table below, it was found that there were positive and statistically significant associations between student agreement that an element was present, with measures of teaching quality, learning resource quality, and learner engagement.

Pearson correlations between perception of the presence of OLM elements and national Student Experience Survey items

OLM Elements national Student Experience Survey (nSES) Measures      
Overall quality Teaching Quality Learning Resource Quality Engagement  
Learning Communities .388** .445** .384** .462**
Interaction between Students .292** .430** .350** .660**
Teacher Presence .712** .796** .681** .374**
Interaction with the Professions .267** .299** .301** .393**
Flexible & Adaptive Learning .696** .716** .662** .413**
Interactive Resources .709** .683** .732** .410**
e-Assessment .715** .710** .730** .353**
** Correlation is significant at the 0.01 level (2-tailed).* Correlation is significant at the 0.05 level (2-tailed).        

As shown in the table there are strong positive correlations between Teacher Presence and Teaching Quality(0.796), and between Interactive Resources and e-Assessment and Learning Resource Quality (0.732 and 0.730 respectively). Correlations between the students’ perceptions of the existence of Learning Communities (0.388), Interaction between Students (0.292), and Interaction with the Professions (0.267) elements of the OLM and the students’ perceptions of the overall teaching quality were significant but somewhat weaker. This suggests that these elements may have had less of an impact on student perceptions of subject quality, or were less strongly implemented.

Findings & Conclusions

Key findings and conclusions from the pilot evaluation are presented below, followed by a summary of the strategies piloted for each of the seven elements and emergent findings and issues.

  • The OLM elements are overall positively viewed by students and staff and do make a positive impact on student engagement and learning processes, although some elements have been more successfully implemented than others in the pilot.
  • OLM elements are interdependent and do not function in isolation from each other – this has implications for future iterations and the scale up of the OLM.
  • The implementation of OLM elements needs to be considered against the context and student profile of the individual subjects (e.g., where does the subject come in the course, what is the student demographic like?).
  • The available technology is sufficient to implement and support the OLM elements.
  • Although the OLM elements are integrated, the pilot data suggest that Teacher Presencee-Assessment_strategies, and _Flexible and Adaptive Learning have the most noticeable impact on student learning.
  • Learning Communities and Interaction between Students elements require more professional development and considered implementation.
  • Implementation of the OLM elements works best with a team-based teaching approach incorporating educational designers, OLM element specialists, subject coordinators, ALLaN, Library, DIT, Learning Resources Unit staff and u!magine.
  • Longer lead in times are required in future scale ups with sufficient time and workload allocations provided to enable mastery of the technology in addition to the OLM element understanding.
  • The provision of paid development time for sessional staff is needed.
  • Easy access to a number of ‘how to’ resources for students, accessible from the subject site, would facilitate student use of technologies used within the subject.
  • Engagement with the element, Interaction with the Professions, was weak and the subjects which focused on this element faced a number of challenges. Opportunities to implement this important element require further investigation and support.