Further to Dave’s post about grabbing the data, we’ve also had successful sample data from Teesside and De Montfort. Check out Fulup’s blog about Stitching together library data with Excel for more details on DMU’s experience.
5 years of book loans and grades at Huddersfield
I’m just starting to pull our data out for the JISC Library Impact Data Project and I thought it might be interesting to look at 5 years of grades and book loans. Unfortunately, our e-resource usage data and our library visits data only goes back as far as 2005, but our book loan data goes back to the mid 1990s, so we can look at a full 3 years of loans for each graduating students.
The following graph shows the average number of books borrowed by undergrad students who graduated with an specific honour (1, 2:1, 2:2 or 3) in that particular academic year…
…and, to try and tease out any trends, here’s a line graph version….
Just a couple of general comments:
- the usage & grade correlation (see original blog post) for books seems to be fairly consistent over the last 5 years, although there is a widening between usage by the lowest & highest grades
- the usage by 2:2 and 3 students seems to be in gradual decline, whilst usage by those who gain the highest grade (1) seems to on the increase
Initial hurdles – the LJMU experience
Energised by the initial project team meeting, the LDIP team at LJMU set about gathering the required data in order to make our contribution to the project. Having already had a few discussions we were fairly confident that we would be able to gather the required data. We had access to student records and numbers, Athens data, library usage data from Aleph and we were aware that our security system (gate entry into the library) kept a historic record of each individual’s entry and exit into the buildings which are serviced through the swipe card entry system. We just needed to pull all this together through the unique student number.
Getting this particular bit of data from our Acadmic Planning department was straightforward. An anonymised list of all 2010 graduating students, along with their programme of study and degree classification, and most importantly their unique id number was duely provided.
Hurdle one – Our security system does indeed log and track student movement through the use of the swipe card entry system, but we are unable to get the systemt to report on this. All data from the system is archived by the system supplier and is subsequently not readily available to us. This means that entry into the Learning Resource Centres is not going to be something we can report upon on this occassion.
Hurdle two – Our Network team systematically delete student network accounts upon graduation, which means the record that links an individual’s unique student ID number, Athens account, security number and Library barcode is not available for the students whose library usage we wished to analyse!
There were about 4,500 students who graduated from LJMU in 2010 with undergraduate degrees, but unfortunately, by the time I got to speak to the network manager, 3,000 of these had been deleted, as is our institutional practice and poilicy.
The upshot of all this is that we are only going to be able to provide data for a third of the potential students that we could have provided data for if we had thought to ask these questions earlier on. But at least we are still able to contribute.
Focus Groups – I am hoping that the organisation and co-ordination of some student focus groups will be more fruitful, but early indicators suggest that the timing of this is not particularly good as we are now in a reading week which will be followed by end of semster exams and coursework submissions, along with an Easter Bank Holiday weekend and Royal Wedding to be squeezed in. In effect, this is the busiest time of the year for our students. However, we have agreat relationship with our student union and they are normally very helpful and responsive so I am hoping we will have something organised very soon.
What would we do differently? – the lessons learnt in this instance are to do with internal partnerships and communication. When first approached about the project we thought that we had asked the right questions of the right people within the University. However, it is obvious to us now that we should have made sure that we discussed our plans in more detail with the Head of Networks and the Head of Security as they are our means of access to two of the key systems that we require in order for us to obtain the required data. Discussions with key stakeholders are of the utmost importance as they highlight local practices and procedures as well as potential difficulties with systems and contracts (as is the case with our security system)
On a positive note all our stakeholders our excited to be involved in the project and do wish that we could provide more data. Our networks manager has already indicated that he would be happy to delay future network account deletions if we wanted to obtain similar data for our 2011 graduates.
To sum up, an interesting couple of weeks at LJMU in our quest to get the LIDP data, and I hope that this post brings with it a few words to the wise……
Hypothesis musings.
Since the project began, I’ve been thinking about all the issues surrounding our hypothesis, and the kind of things we’ll need to consider as we go through our data collection and analysis.
For anyone who doesn’t know, the project hypothesis states that:
“There is a statistically significant correlation across a number of universities between library activity data and student attainment”
The first obvious thing here is that we realise there are other factors in attainment! We do know that the library is only one piece in the jigsaw that makes a difference to what kind of grades students achieve. However, we do feel we’ll find a correlation in there somewhere (ideally a positive one!). Having thought about it beyond a basic level of “let’s find out”, the more I pondered, the more extra considerations leapt to mind!
Do we need to look at module level or overall degree? There are all kinds of things that can happen that are module specific, so students may not be required to produce work that would link into library resources, but still need to submit something for marking. Some modules may be based purely on their own reflection or creativity. Would those be significant enough to need noting in overall results? Probably not, but some degrees may have more of these types of modules than others, so could be worth remembering.
My next thought was how much library resource usage counts as supportive for attainment. Depending on the course, students may only need a small amount of material to achieve high grades. Students on health sciences/medicine courses at Huddersfield are asked to work a lot at evidence based assignments, which would mean a lot of searching through university subscribed electronic resources, whereas a student on a history course might prefer to find primary sources outside of our subscriptions.
On top of these, there all kinds of confounding factors that may play with how we interpret our results:
- What happens if a student transfers courses or universities, and we can’t identify that?
- What if teaching facilities in some buildings are poor and have an impact on student learning/grades?
- Maybe a university has facilities other than the library through the library gates and so skews footfall statistics?
- How much usage of the library facilities is for socialising rather than studying?
- Certain groups of students may have an impact on data, such as distance learners and placement students, international students, or students with any personal specific needs. For example some students may be more likely to use one specific kind of resource a lot out of necessity. Will they be of a large enough number to skew results?
- Some student groups are paid to attend courses and may have more incentive to participate in information literacy related elements e.g. nurses, who have information literacy classes with lots of access to e-resources as a compulsory part of their studies.
A key thing emerging here is that lots of resource access doesn’t always mean quality use of materials, critical thinking, good writing skills… And even after all this we need to think about sample sizes – our samples are self-selected, and involve varying sizes of universities with various access routes to resources. Will these differences between institutions be a factor as well?
All we can do for now is take note of these and remember them when we start getting data back, but for now I set to thinking about how I’d revise the hypothesis if we could do it again, with a what is admittedly a tiny percentage of these issues considered within it:
“There is a statistically significant correlation between library activity and student attainment at the point of final degree result”
So it considers library usage overall, degree result overall, and a lot of other factors to think about while we work on our data!
Notes from the first meeting 11.03.11
The group had its very first meeting on Friday the 11th, and it was a full house – almost all the group members managed to make it to Huddersfield, and were greeted with hot cross buns and biscuits a-plenty.
Introductions were made, and the meeting kicked off with Dave Pattern providing an overview to the background of the project. The germ of an idea began when the library started investigating the kind of people who were using the library, looking at an overall picture rather than something specifically course based. However, it became obvious that there were certain courses who used the library a lot, and some who barely entered, if at all. Creating a non/low usage group within the library at Huddersfield gave the team a chance to focus on targeting specific groups to examine use in more detail, but never created a statistically sound basis to make assumptions, and so the LIDP was conceived!
Graham Stone, the project manager, went through the project documentation and how information is to be disseminated via the blog (with comments welcome from all project members), and reminded members that we don’t consider a positive correlation between library use and attainment to be a causal relationship! The group is very aware of other factors that come into attainment and is by no means suggesting that library use is the only element of importance! Data protection and ethical issues were considered, keeping in mind pending information from Huddersfield’s legal advisor.
Graham asked for volunteers to join a project steering group based at Huddersfield (taking travel distance into consideration!), and it was agreed that Salford would have a representative join the group (a blog post dedicated to the steering group is coming soon).
Bryony Ramsden, the project research assistant, talked about issues that might disrupt the hypothesis (see the main hypothesis blog post), and introduced the idea of running focus groups. Some qualitative data would help explain exactly why some people use the library a huge amount, and some don’t, and help discover why discrepancies between courses might develop. Samples would ideally be a mixture of student types, covering the main groups of undergraduates and postgraduates both full and part time across various schools/bodies. Groups will need to run soon to ensure students aren’t disrupted too much before exams and assignment due dates begin to take up their time, and having found term differences between institutions already the plan was modified from running groups in April and May to over March and April! Data collection could end up running a little tight here, but a move forward could actually be beneficial to all parties if the data is ready earlier than planned.
Dave talked about data collection and emphasised that he realises not all institutions will be able to provide all same sets of data types. He talked through different routes of accessing data to maximise what could be available with a minimum of difficulty. He offered a number of options for passing the data back to him (SQL, Excel, or he can provide coding to help if required), with at least data from academic year of 2009/10. Concerns were expressed that because of variations in graduation dates data may not cover a full academic year, but if these courses are flagged up there may be potential for comparison between like courses. Dave said he’ll create a document detailing the systems of each institution so that he can offer advice easily on data gathering, and reminded everyone that if they have any other data they think might be useful, he’ll welcome suggestions. Data encryption issues were discussed to emphasise the data protection issues raised in the exchange process. Data should be submitted to Dave by 23rd April.
Having discussed all the core important elements to get things moving, the group went their separate ways, some to trains and car journeys, others to the pub (the Head of Steam, right on the train platform for convenience…).
Project plan now available
The latest version of the project plan was aggreed at the last project group meeting. You can find it here.
Our first project team meeting
On Friday the 11 March, all 8 project partners met for the first time. In a packed agenda we discussed the project in detail – we’ll be blogging the minutes soon.
We also approved the project plan and discussed the hypothesis in some detail – look out for our first blog on that soon too! We are now working on getting the focus group questions out to everyone in the next few days.
The meeting went well, and it was good to meet up face to face before we really get started on gathering the data, we started under a pile of biscuits and ended with a civilised drink in the Head of Steam at Huddersfield railway station before the long journey home for some.
Welcome!
This is the blog for the Library Impact Data Project, which is part of the JISC Activity Data programme. The project will run from 1 February 2011 – 31 July 2011.
In addition we’ll be tweeting using the #lidp hashtag and will be archiving it at: http://twapperkeeper.com/hashtag/lidp
Summary
This project aims to prove a statistically significant correlation between library usage and student attainment. Using activity data from three separate systems and matching these against student records which are held in a fourth system, this project will build on in-house research previously undertaken at the University of Huddersfield. By identifying subject areas or courses which exhibit low usage of library resources, service improvements can be targeted. Those subject areas or courses which exhibit high usage of library resources can be used as models of good practice.
The partner Universities represent a cross-section of size and mission and will provide a rich data set on which to work.
Hypothesis
There is a statistically significant correlation across a number of universities between library activity data and student attainment
Project Partners
- University of Huddersfield
- University of Bradford
- De Montfort University
- University of Exeter
- University of Lincoln
- Liverpool John Moores University
- University of Salford
- Teesside University