Excellence Gateway
This case study was produced by JISC RSC (Regional Support Centres) South East on behalf of the Excellence Gateway.
Published: 11 September 2009
Sector relevance: Further education and Sixth Form colleges
Keywords: Improving teaching and learning, improving responsiveness to learners, improving institutional effectiveness, planning and resources, curriculum planning, virtual learning environment (VLE), benchmarking, blended learning, creating and adapting e-learning materials, personalisation of learning, scheme of work, distance learning, giving feedback to learners, motivating learners, formative assessment, achievement rates, attainment, learning outcomes, quality management, staff development, quality assurance policy and procedures
How do you answer the question: “How well is your VLE being used?” Southampton City College rose to the challenge with the creation, development and implementation of a highly motivational ‘Gold, Silver and Bronze' benchmarking system to evaluate and recognise quality of content and activity on its College Moodle system. The results are impressive.
City College is a further education college in the centre of Southampton, offering high-quality courses at every level to students of all ages. Achievement is high, with adult students achieving a 95% pass rate across short courses in 2008. Full-time students study a wide range of courses including BTECs, NVQs and GCSEs. City College understands the importance of career-building and has excellent links with local employers through its work-based learning brand, City Training, the largest provider of apprenticeships in the South.
City College is currently in the final stages of a £35 million redevelopment project with brand new facilities for engineering, hair, beauty, catering, media, performing arts and technical theatre.
When John Savage joined Southampton City College as the new Learning Technologies Manager, his first task was to oversee the migration of the College virtual learning environment (VLE) from Blackboard to Moodle.
As he co-ordinated training for the new Moodle platform within each department, it became very clear to Savage just how the College VLE (called CityBit2) was being utilised. As he recalls:
"The vast majority of staff were using the VLE as a passive repository rather than for creative interactive content…and Moodle works best when users are doing things with it rather than simply retrieving data."
At the same time, the College was formulating its five-year plan to raise standards. In line with its mantra "Outstanding by 2012", Savage was asked to develop a method of measuring the quantity and quality of Moodle usage within the Southampton City College.
An earlier Ofsted inspection gave Savage the idea of using a grading system:
"I didn't want to use the Ofsted grading terminology, as I felt it wasn't motivating. So we came up with the idea of awarding Gold, Silver and Bronze medals instead. The timing was perfect as the nation was building up to the summer Olympics in Beijing."
As a result, all courses on the College Moodle have now been graded in one of four categories:
Every Moodle page is tagged with an appropriate Gold, Silver, Bronze or ‘In Development' icon that is only visible to staff to avoid any stigma.
The implementation of the Gold, Silver and Bronze programme has been influenced by an Ofsted report on VLEs (Virtual learning environments: an evaluation of their development in a sample of education settings) that highlights the lack of institution-wide benchmarking of learning platforms. Based on this report, Savage implemented five phases:
The benchmarking initiative was communicated to the staff through a series of 'tongue in cheek' posters in the College newsletter based on the Olympic Games theme. In the early phase, additional incentives were used. Savage recalls:
"Initially, tutors that submitted their Moodle work were rewarded with MP3 players in order to identify and consolidate the crucial early adopters."
Reports are now taken from the College Moodle and imported into a custom-designed sophisticated spreadsheet, which combines several weighted measurements to come up with a final grade. For each course, these metrics include:
Image 1: Screenshot of the Gold breakdown page on the College Moodle VLE
Image 2: Screenshot of a page in the Moodle VLE showing the medals summary by department
Image 3: Screenshot of Gold CMI course
At the time of writing, the Gold, Silver and Bronze benchmarking system has been in place for one full academic year. The table below shows the improvement in grades from 2008 to 2009:
The total number of awards represents approximately one third of all the ‘genuine' courses on the College VLE. In order to maintain instant feedback, staff are able to see a tally of all the Gold, Silver and Bronze courses on the whole CityBit2 system online at any time.
Savage is delighted with the way the benchmarking scheme has progressed:
"The whole system becomes its own driver. Once tutors receive a Bronze award, they start coming to me asking what they have to do to promote it to Silver or Gold. As a result, we now have a larger proportion of courses that are making the most of Moodle's sophisticated facilities in the form of interactive and collaborative content that the students find much more engaging."
In order to further encourage staff to improve their Moodle material, tutors can click a link beneath the Gold, Silver, Bronze or 'In Development' logo, which provides detailed feedback for that course, cross-referenced to the Gold, Silver and Bronze criteria. This allows staff to see, at a glance, what they have to do to progress to the next medal. Additional online assistance is also available at the click of a mouse.
The spreadsheet used to transform raw Moodle reports into a weighted grading score is quite complex and took around 40 man hours to produce. However, it now allows almost completely automatic grading with only a tiny proportion of courses requiring manual moderation.
Savage does raise the issue of targets:
"If your VLE benchmarking scheme is part of an exercise to achieve targets, be very careful just what you count as a ‘genuine' course. Temporary ‘overnight' Moodle courses thrown together quickly, or those being used as some kind of sandpit, will drag your statistics down. Stipulate some minimum requirements - such as a minimum enrolment number – before you allow a course to be included within your benchmarking statistics."
Additionally, Savage advises the following:
As the Gold, Silver and Bronze benchmarking scheme enters its second year, Savage is constantly looking for improvements. His ‘wish list' includes:
For Savage, the two most crucial future developments that will have the biggest impact on the College VLE benchmarking scheme will be the introduction of single log-ons for students and the implementation of a streaming media server. He summarises:
"These two initiatives would help enormously by bolstering the scheme at two opposite ends - combining more engaging content with easier access for the students."
By 2010, the College is looking to achieve 70% of courses in the medals with 10% Gold, 25% Silver and 35% Bronze awards. Only time will tell, but on present form Southampton City College may find its Moodle medal haul puts the GB Olympic Team to shame.
TIP: Have you rated how useful this case study is? Use the star-rating facility, which features in the grey bar that runs across the top of every page of this site. (To rate a case study, or any page in the Excellence Gateway, you need to log in to the site first.)
Disclaimer: The Regional Support Centres (RSC) and the Learning and Skills Improvement Service (LSIS) support the development of educational e-learning. In the case study, we may refer to specific products, processes or services by trade name, trademark, manufacturer or otherwise, or link to websites or supporting material. Such references are not endorsements or recommendations and should not be used for product endorsement purposes.
You can find this page and download any referenced resources from the Excellence Gateway at http://excellence.org.uk/245895.
