16 March 2009 1 Comment
A couple of weeks ago, I travelled up to Edinburgh [photos on Flickr] for a meeting to discuss the JISC funded EVAF4All: Electronic Voting Analysis and Feedback for All project. Led by Simon Bates and Keith Brunton from the e-Learning unit in the School of Physics and Astronomy at the University of Edinburgh, colleagues from Imperial College London, City University, the Universities of Bath, East Anglia, Leicester and Manchester were in attendance, along with a representative from the JISC-RSC for Scotland.
The current EVAF system allows students to obtain a record of their progress in formative assessment using an Electronic Voting System (refered to an Audience Response System, ARS, at the University of Bath). When viewing their results, students will be directed back to relevant content and presented statistics regarding whole class performance. This feedback can thus aid their developing self-management of their own studying and provide valuable diagnostic feedback. The staff view of the system will enable aggregated data to be extracted and analyse aggregated student performance. [see EVAF website for further information].
The aim of the EVAF4All project is be create an EVS-neutral solution the application to be installed at a variety of institutions irrespective of the VLE and EVS handset (“clicker”) loan scheme. This process in turn would in turn close the ‘feedback loop’ (see Nicol and Mcfarlane-Dick (2006) for further information about this concept) for students engaged in learning activities.
Over the course of the day, a variety of topics were covered in relation to moving the project forward. I have divided these conversations into three main areas, as follows.
1. “Loanership” schemes
The conversation moved onto the different forms of clicker “loanership” scheme currently employed at the different institutions. At my own institution for example, 200 TurningPoint clickers are held by a central AV team. Bags of clickers can be booked out in multiples of 40 clickers [and a single USB RF receiver]. Generally speaking, student participation in formative assessments are completed anonymously.
Other institutions had developed a variety of mechanisms for supporting EVS-related work within learning and teaching. For example, at the University of Edinburgh, students borrow the handsets from their library, where the handset is lent out using a similar process for when taking out a book. Once the barcode on the back of the handset is scanned, the library system, the students records system and the VLE (as well as the EVAF software!) all know that the student has that particular handset. This in turn makes tracking student progression that much easier. The University of Edinburgh currently have over 2500 handsets, across a wide range of disciplines, which are loaned out in such a way. Students can then track their progress using the EVAF application.
In the School of Biological Sciences at the University of Leicester, first year students are lent a handset for the academic year in return for a £10 refundable deposit – the overall loss rate in the first year was 10% of handsets broken or not returned (the latter mainly from students who dropped out). The handsets have been used in the past to track student attendance, meaning that a clicker can be tied to a particular student.
It is because of such loanership models, that these two institutions are in a better place to use their existing e-Learning/MIS infrastructure to participate in the EVAF pilot, and thereby provide a case study of usage. That said, the University of Bath could do so too… it might just take a little longer to get to the end point, and require a little more human intervention to make the process work. [I'll blog about this further as the project progresses.]
2. Technical infrastructre
As I alluded to in the section above, the are a number of technical requirements that will need to be addressed before the University of Bath will be able to connect to the relevant EVAF instance. For example, at the University of Bath, we have the following systems: Moodle (our VLE), SITS (the student records system), TurningPoint (the EVS system). I would expect that these details will vary across the institutions involved in the project currently. Both Simon and Keith stressed that they will make best efforts to integrate EVAF into our existing structures, but given the timescales, this may be tricky without additional support at a local level. Thankfully, the e-Learning team at the University of Bath have two excellent and enthusastic software developers in house!
3. Getting feedback to students
The key element of this project is of how to best get feedback back to students once the EVS has been used in a lecture. Whilst initial, whole class feedback can be give within the assessment context (e.g. the lecture theatre), the EVAF application seeks to allow students to review the questions post-lecture in their own time. Aligned with this, it is about stressing the importance to students of formative feedback in the learning process. For staff, the software provideds for an easy and effective way to present and view large volumes of user data in a simple way, and begin to assess the effectiveness of the questions used.
Using the TurningPoint 2008 software, colleagues at the University of Bath can already make some question data available to students in a proprietry format, which the students can then analyse themselves. However, this is whole class feedback rather than individual feedback. EVAF takes this process further by personalising feedback and giving the individual student the opportunity to examine their strengths and weaknesses.
At Bath, where no loanership scheme is in place, we still have the problem of how to make this individual feedback available to students (and staff) in an easy and quick way, without a significant additional overhead of time or resource.
I am encouraged by the successes of the EVAF at the University of Edinburgh to date, and applaud both Simon and Keith for having the vision and committment to attempt to roll this project out to additional partner institutions. In addition, both were very receptive to feedback about the EVAF application, and I look forward to becoming more involved with the project as it develops.
I have no doubts that EVAF would be a success if it was implemented at the University of Bath, and in turn, used by the students. I would hope that it would have an impact on any surveys such as the NSS, where students often complain about the lack of feedback from assessment!