A colleague here at UNSW Kate Coleman has waxed lyrical about a new tool that has been developed here in Sydney at UTS and is being trialled at UNSW. It is called ReView and looks very promising. The real strength in this tool for the Australian HE sector seems to lie in its capacity to link through to Graduate Attributes, which perform a similar function to bench-marking in the UK. You can see a list of Graduation Attributes at Australian Universities here. The tool requires markers to nominate and mark to a set of learning outcomes or assessment criteria – something that other tools such as Grademark don’t make mandatory. This may sound limiting, but could potentially be a change-management catalyst if used in the right way. First impressions of the tool seem to suggest that it’s going to be particularly useful for assessments other than standard essay submissions: things like art works, performances (in music or drama for instance) where there is a product which the student generates that needs to be marked. This is especially so given that  Re:View works really well on tablets such as iPads. I’ve been offered training on the tool by someone whom Kate assures me is an evangelist. She told me that a lot of comparative analysis has been done of the tool against competitors like Grademark, so I will be interested to see their findings on that one. There is a page on the TELT Gatewaythat offers a case study on its use which is worth looking at.  I’m hoping this may provide us with a missing piece of the EAM puzzle. I’ll report further when I’ve got my mitts on a live version.

The University organised an Arena 51 afternoon, devoted to improving the Student Experience on Tuesday 10 January 2012: 1.00pm – 4.00pm. this was an ideal opportunity to share some of the preliminary findings of EBEAM with colleagues. I delivered a session explaining how we were using EAM in the School of Education and Professional Development and here’s a summary of the points raised in the consequent discussion…

Arena 51: The Student Experience Event

Session Title: Benefits of electronic assessment
Name of Convener: Cheryl Reynolds

Summary of conversation
Benefits:

• Less time collating work to mark
• Less chance that work is lost
• Submission doesn’t have to be f2f – convenience for distance students
• Sustainability (less paper)
• Quick marks (pre-written comments for common errors) free up time and cognitive space to give better feedback.
• Cohort wide data flags up where students are weak and this informs curriculum design.
• Sharing this data with the students helps them see their norm-referenced achievements and assures them of the importance we place on the assessment process.
• Illegible handwriting avoided
• And lots more

Caveats raised include:

• Difficulties presented to those members of staff who don’t touch type
• Preference for paper-based processes
• Investment of time needed at the outset to get to grips with the tool
• External regulatory bodies stipulations that preclude the kind of assessment that can be assessed through GradeMark (though it was pointed out that any type of assessment can be graded in GradeMark if the correct submission process is used.)

Wish List includes:

• Audio within GradeMark, though there is a way to do this through Unilearn (not GradeMark) described by Steve Bentley

Actions
Staff development session – hands on use of GradeMark Sue Folley is willing to organise this as a follow up to an earlier session that demonstrated GradeMark to staff.

Not agreed in the meeting but perhaps Steve would be willing to contribute some input on how to attach audio to UniLearn feedback, either to this session or to a separate one?

Feedback provided to Helen Walker in Staff Development who will convene meetings and make arrangements for the above actions.

I’m now in Sydney, settled in at the Learning and Teaching Unit at the University of New South Wales (I’m babysitting the office of Adele Flood). It’s great to be here and meeting some of the folks who work here, but it’s also interesting to discover that UNSW, like the University of Melbourne, are still heavily invested in paper-based assessment management systems. It seems that UNSW has a lot of different systems being used across different Faculties to provide feedback to students in interesting ways, but for the most part paper-based submission, marking and return seems to prevail. I’m growing increasingly convinced that the problem we are facing in the UK of needing to move towards Electronic Assessment Management is more widespread than I had originally envisaged.

The UNSW Teaching Website has a particularly rich set of resources under the heading of ‘Assessment‘ which are consistently focused on reinforcing the message about assessment as learning. I really like this turn of phrase (as distinct from assessment of learning and assessment for learning) and as Cheryl mentioned in an earlier post, was something Adele Flood used when she came to Huddersfield to speak last year.  The help sheet on Using Assessment Rubrics also caught my eye as being particularly useful, balanced and helpful and has particular application to building and using rubrics within Grademark. This chimes with one of the toolkits we’re preparing as an output for this project.

I’m looking forward to picking the brains of the good folks at this fine institution, but also hope that some of the ideas and experience we’ve generated as part of the EBEAM project can be of use to those who want to make the move away from paper-based assessment management methods here.

It was great to meet up with Robyne Lovelock from ALDIS who represents Turnitin in Australia and the Pacific, and Travis Cox from the University of Melbourne today for a spot of pizza in Lygon St – the famous restaurant strip in Melbourne’s inner city. We spent a good hour or so talking electronic assessment management over pizza and pellegrino.  Some reflections on our conversations:

– Travis shares with me the concern that investing in home-grown tools to manage assessment can be a risky approach. The ongoing costs of sustaining these tools in the face of the development of proprietary systems is always going to be hard to maintain. And once an institution has invested (both financially and emotionally) in a tool that they’ve built, they become all the harder to jettison even if something better comes along. Making a decision to use proprietary tools for which institutions already hold licenses and finding ways to join them together (with work arounds and by making adjustments to regulations) is likely to be a more sustainable strategy than building something from scratch that does everything.

– I mentioned a problem we were facing with one of our institutional regulations and he suggested a potential solution that he knew about because he has administrative access to the Turnitin building block. It just goes to show that sharing experiences and queries between institutions is always going to be beneficial. This tiny piece of information will potentially save Huddersfield a lot of time and effort over the long run while also ensuring that our academic conduct procedures are more robust.