Category Archives: Users

LIDP Focus Group Write-Up

Martin Philip (Subject Librarian for Accountancy, Law and Informatics) reflects on a focus group held with a group of our ‘non users’ from the School of Computing and Engineering.

I recently conducted a focus group with five Computing students, a department that has been identified as low users of the library.

The focus group was conducted using the ‘flipchart paper and post-it notes’ format, as designed by Ellen. There were five parts to the session, beginning with an activity that asked students to explain where they got information that was used in their last assignment. Questions asked included ‘what was the information?’ and ‘how did you find it?’

Initially, the students were asked to think about their last assignment and answer questions including ‘What was the information you were looking for?’, ‘How did you find the information?’, ‘What format did the information come in?’ The students were asked to discuss their answers with one another and then they were then to write down the answers onto orange post-it notes. The students had to then put the post-it notes onto a piece of flip chart paper which had a scale on it from ‘use lots’ to ‘never use’.

Examples of answers students wrote down near the ‘use lots’ end of the scale included Google, Internet, MSDN, Tutorials Online, UniLearn (VLE) and E-books. At the other end of the scale, students said they rarely spoke to third years, when looking for information, nor did they consult suggested module reading or use many books.

fg1

The students were then given lots more post-it notes to place on the scale with the difference being that these were pink and had already been written on with examples of resources. As before, I asked them to discuss and then place on the scale. Google, Websites and YouTube were examples of resources the students rated as ‘use lots’. Resources at the ‘never use’ end of the scale included Journals, Library Subject Enquiry Desk, Total Training, ACM Digital Library and Unitube.

Here’s a photo of the flipchart paper with both sets of post-it notes (students answers in orange, my suggestions in pink). The scale lies on the paper left to right:

I then split the students into two groups, one with three, the other with two, and asked each group to choose one thing from the scale which they rarely or never use. On a blank piece of flip-chart paper, they brainstormed all the reasons that they rarely/never use this specific resource or service.

fg2

After 10 minutes, the students then looked at the other groups ‘brainstorm’ and made some brief comments. I then closed the focus group with an open discussion based on questions they had come up with during the activities.

Discussion

The students began discussing reasons for not using Unitube and Magazines, the resources that each group picked out during the brainstorm session.

The main reasons for not using Unitube were that they do not know what it is, what information is provided on it, or how to get access to it. Many students admitted to noticing the logo being advertised around campus but made assumptions that it wasn’t required for them. They explained that they regularly use YouTube which they find easier to navigate than Unitube.

The students didn’t make use of the university-subscribed magazines because they felt like they could find the same content online but in a quicker, easier way than visiting the library. They didn’t feel like there was any benefit to using magazines to find information for their assignments. “Using magazines is going out of your way to use it when instead you can just Google it.”

A key theme of the discussion that was repeated throughout was a sense that the students would only use resources that they were told to use by their lecturers and didn’t see the benefits of using other resources. “We don’t see the point (in using x) because we’re not told to use them” one student said. Another added “We’ve not been told about certain things so we just stick with what we know.”

This particular sample of students gave some reasons why they don’t tend to use the university’s paid-for resources. “The thing is, we’re on courses that don’t require us to use some resources that are heavily book-based. We can just find everything online. Other courses do a lot more essays and have to reference things, when we do more practical work so using video tutorials is better than finding references in books.”

Other reasons for not using resources include a lack of awareness.  “I’ve no idea where the Library Enquiry Desk is and I’m a second year.” (This refers to the help desk that is staffed by librarians.)

“For the ones (resources) we don’t use much are I think we’re not aware or there is another alternative we use instead or they’re not really relevant.”

A number of students talked generally about where they get their information from. One explained “You have your preferences and everyone is different but you have stuff that you have grown to use something so if it works I’ll keep using it, why use something else if you’re getting on fine doing what you’re doing?”

Another agreed detailing “For me, its books and Google. They’re the two main places I go. If I know things are in them, I’ll keep going back because I get into the habit of it. If it’s been published it’s got to be decent. I don’t like to venture out.”

And finally, another student explained that it was about location, “For me it depends where I am. I do use Summon a lot when I’m at home, but if I’m in university I prefer using the library and getting a book.”

Lots of reasoning was to do with how ‘easy’ a resource was to use rather than a recognition of quality. When talking about why the students preferred to use the ‘open’ internet, one student explained “With stuff that’s provided by the uni and that, none of us have used them before we came here so we just stick to what we know.” The students didn’t seem to have a desire to search for information beyond what they are told.

Some students found the session helpful, explaining “I think it’s good to know that there is more to use than what I’ve been using to find stuff. So if it’s not on the things that I use, it’s good know there are some alternatives.” There wasn’t the recognition that the university-subscribed resources were providing superior content to the free online resources the students preferred to use.

One student did conclude that there was merit in using the university-subscribed resources adding “For me I hadn’t heard of Digital Tutors until I came to university because you have to pay for it. I know that Digital Tutors, EatsD and 3dmotive are just three of the resources that we have access to through the university. I never used these before because you had to pay for them. It is something I use a lot.”

Further analysis

We are pulling out much more detailed data for LIDP2, including the time of day a resource was accessed. Some initial graphs are now available here, however, please note that this is just raw data and need to be cleaned up.

There do seem to be some interesting trends. For example the graph below implies that not only do students who achieve a 1st class degree use more e-resources, but they also use more during ‘office hours’, students who receive a lower degree are the highest uses overnight – this appears to tally with other research that shows that students who log in to a universities VLE are more at risk of dropping out.

The next graph appears to show that students who obtained a first use e-resources most between 2am and 7am – perhaps chasing deadlines? Students who get a 1st peak btween 10am and 11am.

Please note that we haven’t cleaned this data up yet, e.g. we haver not excluded overseas users.

Talking to Business Librarians at the BLA Conference

We have been out and about disseminating the early findings of the LIDP project over the last few weeks. We have been delighted with the feedback we have received from conference delegates and a lot of the comments about possible future directions for research from the CILIPs, SCONUL and LIIBER conferences have given us food for thought. Many of these comments will appear in the final project blog post before the end of July. However, we had the opportunity at the Business Librarians Association Conference at Sheffield (http://www.bbslg.org/2011Conference.aspx) of testing some of these thoughts. After our presentation (http://eprints.hud.ac.uk/10949/) we divided delegates up into a number of groups to discuss a variety of scenarios.

Scenario 1
If we assume a link between library usage and attainment, what does good practice look like? What are the students who gain a first doing differently to their colleagues who get lower grades? Do high achievers choose ‘better’ resources, or are they ‘better’ at choosing resources?
Two groups reported back on this scenario with the following recommendations:

  • Talk to high achievers to find out what they are doing, e.g.
    • Working
    • Using data effectively
    • Using the right resources
  • Establish what good practice is, e.g. finding, using interpreting
  • Consider the requirements of the subject, for example mathematics courses often require much less resource use than other subjects such as history
  • Qualitative statistics need to be considered in addition to quantitative statistics
    Consider the impact of information literacy and support services
  • Find out the student’s own personal goals, e.g. why are they attending the course – as a work requirement etc.
  • Look at which resources are being used, such as extended reading, not just how much
  • Teach the students evaluation skills to help them find appropriate resources, not just ‘better’

Scenario 2
If students are not using the library or the resources, what can we do to change their behaviour? Is non-use a resourcing issue or an academic/information skills issues? How could gender, culture and socio-economic background affect library usage and how could this be addressed? Are there scenarios where we should NOT try to increase library use?

Groups considered a number of factors that could be used to change behaviour:

  • Incentives
    • Attached to an assignment
    • Work with and win over the academics
    • Encourage student champions
    • Make sure the resources are embedded and relevant to the subject

Regarding non-use, the groups thought that both issues were relevant. The skills issues required further training and the resources needed simplifying.
Gender, culture and socio-economic background were themes brought out at both the SCONUL and LIBER conferences. One group looked at international students where it was considered that they were too dependent on Google – does this means our resources are too difficult to understand? It was also considered that there is a focus on generalisations, e.g. international students, rather than looking at individuals. Another group considered that it was a cultural issue and that students were guided to the ‘right answer’ via reading lists, rather than reading around the subject.
Finally discussion turned to work-life balance and whether students should be logging in at 2am, and whether our culture of 24×7 access was a healthy one.

Scenario 3
Can we actually demonstrate that the library adds value? E.g. if a student enters university with average UUCAS points and attains a first class degree having used library resources to a high level, does this prove the library has added value to the student achievement? Have we done anything? Do they need us?

The short answer to this scenario was yes!
We receive feedback, both internal and external and have provided learning spaces and essential resources at the very least. We can also show that we have promoted our services and embedded information literacy skills into the curriculum by working successfully with academic staff. It was thought that we add to the employability of students by teaching them research skills and giving certification, e.g. Bloomberg etc.

Scenario 4
If the hypothesis is proved to be correct, does cutting library budgets mean that attainment will fall? Is this something that can be used at director level to protect resource budgets/subject librarians? Should we be concerned about implications for publishers if the hypothesis is proven?

The group that looked at this scenario considered that further use of statistics were required to find out what students were reading. This would allow stock to be rationalised and the reduced budget could be used to better target appropriate resources.

In addition it was suggested that other services such as, inductions and information literacy training by audited and evaluated in order to provide more effective targeting.

It was also felt that there was an absolute minimum spend for resources, once this level was passed impact would be huge with insufficient resources to support courses.

The group felt that this could be used at Director level and that evidence would be required to support this.
Big deals came up in the final point from this scenario. Discussion centered on a standoff between the need for better products verses ongoing financial commitments

Many thanks to all the delegates for allowing us to blog about their comments and to the BLA for letting us loose at their conference. We’ll be adding some of these comments to our final blog post.

What will this project do for library users?

The project aims to make some pretty big conclusions by the end of the data analysis about library usage and attainment, but what can we actually do with this information once we’ve got proof?  What use is it to our customers? 

We’ve got two main groups of library users; staff and students.  We aim to use our quantitative data to pinpoint groups of students who have a particular level of attainment.  We’ll work with staff in order to improve poor scores and learn from those who are awarded high scores, regardless of whether they are high or low users of our resources and facilities.  Focus groups held now, and most likely regularly in the future, will tell us more about people who use the library resources less but achieve good degree results.  If the materials we are providing aren’t what students want to use, we can tailor our collections to reflect their needs as well as ensure they get the right kind of information their tutors want them to use.

The student benefits are pretty obvious – the more we can advise and communicate to them and encourage use of library staff, and electronic and paper resources, the more likely they are to get a good degree and get value from their time (and money!) spent at university.  Once again we state here that we are aware of other factors in student attainment, but a degree is not achieved without having some knowledge of the subject, and we help supplement the knowledge communicated by lecturers. 

Students get value for money and hopefully enjoy their university experience, lecturers ensure students get the right kind of support and materials they need, and we make sure our budget is used appropriately.  Pretty good, huh?

Some thoughts on the focus groups from De Montfort University

Given the time of year and the short time scale in which to hold them (just before Easter), we were pleasantly surprised to receive an overwhelming 204 replies to our invitation email for the three LIDP focus groups. The ten pounds print credit incentive must have looked particularly attractive during assignment time, especially when our most generous offering for focus groups so far had not exceeded five pounds.
Expecting less than 50% attendance, we invited twenty students to each focus group and gently let down the rest. Attendance was also better than expected with thirty five students attending in total, twenty six of whom were full time undergraduate students.

Students were on the whole interested, enthusiastic and some questions and comments generated lively discussion around the table especially when talking about mysteriously missing files from the library PCs! There were some insightful comments: summing up a conversation about the limitations of some of the library searching tools and some ways around these, one student remarked ‘it seems that a lot of us use different means to go around the library rather than use the library engines as we can’t find things by using them. We are working around the library not through it’

At first, I was not sure what the focus groups could add to the very neat graphs that Dave has already produced from our quantitative data. Students who attend focus groups are not usually a representative sample, and these groups were no exception. As one student remarked ‘we have all made the effort to come to this focus group, it kind of shows we are in the same mind’ (i.e. motivated and keen to do well).

However, even if for a biased sample of the students’ population, the focus groups did flesh out the story behind the figures. What these students have in common is their active engagement with the academic world, including the library and its resources. Most of them read beyond the recommended reading, use the online resources, borrow books regularly, and are keen to get a good degree. This does not mean that they do not get frustrated by faulty equipment and missing books, of course, but they all showed a willingness to make sense of their academic environments, some even finding ingenious ways around the perceived inadequacies of our systems.

It would be expedient to think that it is our wonderful and expensive resources that make the difference in students’ performance and ultimately their results. But I suspect that a more crucial factor is the depth of the students’ engagement with their studies rather than the intrinsic value of our resources. My guess is that most of the students attending the focus groups will go on to do well in their studies. They will do well because they are keen, and because this motivation is translated into a willingness to try things out and explore the resources and services at their disposal.

The fact that many students comment on the awkwardness of our systems and searching tools (i.e. catalogues and databases) could also have a role to play in explaining the correlation between Athens logins and degree results. Motivated students are more likely to explore the resources that are available to them and also more likely to jump over hurdles and persevere to get to the good stuff. So, could the strong correlation between Athens logins and degree results be as much an indicator of students’ motivation and staying power as it is of the usefulness of our resources? And could the advent of discovery tools like Summons or Ebsco Discovery lessen this correlation? Indeed, if searching for ‘quality’ resources becomes as ‘easy’ as searching Google, will usage of online library resources still be a measure of the difference between the good and the not so good student? Or will the difference only become noticeable further along the way (e.g. how students make sense of the information they find). But if so, will we be able to measure it?

The focus groups also helped to explain the lack of correlation found between usage of the library itself (i.e. the physical space) and degree results. Although most students use the library regularly, there is a very clear division between those students who prefer working in the library and those who prefer working at home. This preference does not appear to be linked to motivation or engagement with their course but to other factors such as personal preferences, distance from the library, the nature of the task undertaken, and the availability of internet access at home. So for those students, using the library as a space is not an indication of how hard they work. Moreover, whilst Athens cannot be used for much else besides studying, there are many more ways in which the library can be used than for studying (e.g. using the PCs for fun, chatting, meeting place).

All in all, the focus groups were a great opportunity to meet some great students, gain a deeper insight into students’ experience of using the library, and generated a lot of interesting qualitative data. It also provided me with much food for thought and speculation!

Marie Letzgus
De Montfort University

The legal stuff…

One of the big issues for the project so far has been to ensure we are abiding to legal regulations and restrictions.  The data we intend to utilise for our hypothesis is sensitive on a number of levels, and we have made efforts to ensure there is full anonymisation of both students and universities (should our collaborators choose to remain so).  We contacted JISC Legal prior to data collection to confirm our procedures are appropriate, and additionally liaised with our Records Manager and the University’s legal advisor.

Our data involves tying up student degree results with their borrowing history (i.e. the number of books borrowed), the number of times they entered the library building, and the number of times they logged into electronic resources.  In retrieving data we have ensured that any identifying information is excluded before it is handled for analysis.  We have also excluded any small courses to prevent identification of individuals e.g. where a course has less than 35 students and/or fewer than 5 of a specific degree level.

To notify library and resource users of our data collection, we referred to another data project, EDINA, which provides the following statement for collaborators to use on their webpages:

“When you search for and/or access bibliographic resources such as journal articles, your request may be routed through the UK OpenURL Router Service (openurl.ac.uk), which is administered by EDINA at the University of Edinburgh.  The Router service captures and anonymises activity data which are then included in an aggregation of data about use of bibliographic resources throughout UK Higher Education (UK HE).  The aggregation is used as the basis of services for users in UK HE and is made available to the public so that others may use it as the basis of services.  The aggregation contains no information that could identify you as an individual.”

Focus groups have also been conducted with a briefing and a consent form to ensure participants are fully aware of data use from the group and of their anonymisation and advising them that they can leave the group at any point.

Notes from the first meeting 11.03.11

The group had its very first meeting on Friday the 11th, and it was a full house – almost all the group members managed to make it to Huddersfield, and were greeted with hot cross buns and biscuits a-plenty.  

Introductions were made, and the meeting kicked off with Dave Pattern providing an overview to the background of the project.  The germ of an idea began when the library started investigating the kind of people who were using the library, looking at an overall picture rather than something specifically course based.  However, it became obvious that there were certain courses who used the library a lot, and some who barely entered, if at all.  Creating a non/low usage group within the library at Huddersfield gave the team a chance to focus on targeting specific groups to examine use in more detail, but never created a statistically sound basis to make assumptions, and so the LIDP was conceived! 

Graham Stone, the project manager, went through the project documentation and how information is to be disseminated via the blog (with comments welcome from all project members), and reminded members that we don’t consider a positive correlation between library use and attainment to be a causal relationship!  The group is very aware of other factors that come into attainment and is by no means suggesting that library use is the only element of importance!  Data protection and ethical issues were considered, keeping in mind pending information from Huddersfield’s legal advisor.

 Graham asked for volunteers to join a project steering group based at Huddersfield (taking travel distance into consideration!), and it was agreed that Salford would have a representative join the group (a blog post dedicated to the steering group is coming soon).

Bryony Ramsden, the project research assistant, talked about issues that might disrupt the hypothesis (see the main hypothesis blog post), and introduced the idea of running focus groups.  Some qualitative data would help explain exactly why some people use the library a huge amount, and some don’t, and help discover why discrepancies between courses might develop.  Samples would ideally be a mixture of student types, covering the main groups of undergraduates and postgraduates both full and part time across various schools/bodies.  Groups will need to run soon to ensure students aren’t disrupted too much before exams and assignment due dates begin to take up their time, and having found term differences between institutions already the plan was modified from running groups in April and May to over March and April!  Data collection could end up running a little tight here, but a move forward could actually be beneficial to all parties if the data is ready earlier than planned.

Dave talked about data collection and emphasised that he realises not all institutions will be able to provide all same sets of data types.  He talked through different routes of accessing data to maximise what could be available with a minimum of difficulty.  He offered a number of options for passing the data back to him (SQL, Excel, or he can provide coding to help if required), with at least data from academic year of 2009/10.  Concerns were expressed that because of variations in graduation dates data may not cover a full academic year, but if these courses are flagged up there may be potential for comparison between like courses.  Dave said he’ll create a document detailing the systems of each institution so that he can offer advice easily on data gathering, and reminded everyone that if they have any other data they think might be useful, he’ll welcome suggestions.  Data encryption issues were discussed to emphasise the data protection issues raised in the exchange process.  Data should be submitted to Dave by 23rd April.

Having discussed all the core important elements to get things moving, the group went their separate ways, some to trains and car journeys, others to the pub (the Head of Steam, right on the train platform for convenience…).