Methods:
For the 2023/2024 assessment cycle, the University Libraries assessed Learning and Performance Outcomes. The requirements for annual assessment are for two methods with results. This cycle, the University Libraries completed four methods with results. These are:
Stakeholders:
Information and Media Literacy Module Evaluation (FYS 1010)
Assessment Method:
In fall 2023, the FYS 1010 sections of the Wayne Experience (WE) General Education requirement had an integrated video training that introduces students to information evaluation. This segment is designed to be completed in 50 minutes (the length of a class session for this course). It aligns with the content the University Libraries focused on previously--introducing these concepts and how the library enhances access, introduction to Summon, database searching and Research Guides.
A brief (five-question) evaluation follows this training. We will use three of those questions in evaluating two of our outcomes: Evaluation of Information Sources and Locating Information for Assignments.
This method will supply data for Outcomes 2 and 3.
Method History:
This assessment method has been used before in its current form.
Results :
We had 128 responses. There were 128 complete responses for Learning Outcome 2 and 127 complete responses for Learning Outcome 3. Note that there were significantly fewer responses than the previous year’s 648, due in part to a significant drop in the number of sections of FYS 1010 due to suspension of the sections of the Wayne Experience (WE) General Education requirement.
19 responded to Question 2, “Before you took this FYS 1010 course, how knowledgeable did you feel about evaluating information you found on the internet for reliability and credibility,” with “Not knowledgeable at all” or “Slightly knowledgeable.”
Of those 19 respondents, all 19 responded to Question 4, “Now that you have completed the Information and Media Literacy module, how knowledgeable do you feel about evaluating information you find on the internet for reliability and credibility,” with “Moderately knowledgeable,” “Very knowledgeable,” or “Extremely knowledgeable.”
(0 not knowledgeable, 0 slightly, 7 moderately, 9 very, 3 extremely)
100% gave the desired response of moderately, very, or extremely knowledgeable. The measure exceeded the 80% response that indicates a successful achievement of Outcome 2, Evaluation of Information Sources. Note that the target goal was increased from the previous year’s goal of 75%.
75 of the 127 responses named three or more resources or services.
59% gave the desired response. The measure falls below the 75% response that indicates a successful achievement of Outcome 3, Locating Information for Assignments.
These are the numbers of times each resource or service was named:
|
Look for information in new or different places (including library databases) |
79 |
|
Look at an author’s credentials to better evaluate a source for credibility |
92 |
|
Use a website like Media Bias FactCheck to investigate a source |
72 |
|
Use a library research guide to help get started looking for information |
70 |
|
Ask for help using the libraries using their 24X7 chat |
44 |
|
Nothing |
4 |
Program Action Plan:
Fewer sections of FYS 1010 were offered in the Fall 2023 semester, than in previous years, and we reached significantly fewer students than we anticipated. Due to ongoing changes in General Education requirements for the Wayne Experience, and changes to its curriculum, along with changes in the information landscape, we are reevaluating the content in the self-paced modules to include AI in our information and digital literacy instruction. The Learning Outcome 2: Evaluation of Information Sources pre-test and post-test will be modified for the new content. In addition, for Learning Outcome 3: Locating Information for Assignment, we will evaluate and improve the instruction modules with the goal of improving students’ understanding of the available resources and their benefits.
Timeline for Action Plan Implementation:
The revised instructional materials and assessment method will be used for all sections of FYS 1010 during the Fall 2024 semester.
Library Chat Reference Service
Assessment Method:
The University Libraries are able to provide 24/7 chat reference service by participating in both a state-wide academic library chat cooperative, ResearchHelpNow!, and a global cooperative, 24/7 Academic Global. This important service answers about 4,000 student, faculty, staff and community member questions each year. While most of the questions (60%) are answered by library staff members, librarians from our cooperative institutions are also responsible for a significant part of this service. At the end of each chat interaction, patrons are invited to rate their satisfaction with the service and offer comments on their experience. We will be evaluating the chat reference service based on the patron satisfaction ratings. We will use the embedded chat ratings tool within the proprietary software, LibChat, to collect patron satisfaction ratings of our chat reference service on a scale of 1-4 (1=Bad, 4=Excellent). Our goal is to maintain an average patron satisfaction rating of 3.7 or higher. Data will be collected on a semester basis (Jan-April, May-Aug, Sept-Dec), analyzed, and compared with previous corresponding semesters by the Coordinator for Reference Services and a small team of Librarians who perform chat reference. An equal or higher chat rating average will indicate an equal or higher level of patron satisfaction with our chat reference service.
This method will supply data for Outcome 5.
Method History:
This is a new assessment method that has not been used before in its current form.
Results :
Chat ratings from Fall 2023 and Winter 2024 semester remain steady and above our target threshold of 3.7 or higher. When comparing the average ratings from corresponding semesters of FY24 with FY23, we see strong, consistent ratings.
Program Action Plan:
We will continue to monitor chat reference ratings on a semester-by-semester basis and continue with 3.7 or higher as the target threshold for patron ratings. Our analysis will be shared more broadly with the team of reference librarians at a future meeting.
We see further potential to look at the data specific to chats that are referred to our ticketing system. There is a future opportunity to strengthen the tools that support our chat reference and see if that results in consistent or stronger service levels.
Timeline for Action Plan Implementation:
This assessment method is an ongoing initiative conducted on a semester-by-semester basis. We will share the analysis with library administration and reference librarians during the spring/summer 2024 semester.
Information Literacy Module Evaluation (COM 1010)
Assessment Method:
The University Libraries developed a series of instructional modules on information literacy using a proprietary software, LibWizard, that can be integrated into course content by instructors of COM 1010 during the WI 2024 semester. Integration of the modules is optional, and instructors may choose to incorporate all, some, or none of them. To inform future improvements to and/or developments of library instructional materials, it is important to know which modules instructors use and their observations of impact on student learning. A small team of instruction librarians, led by the Student Success Librarian, will analyze the perceived impact of the embedded COM 1010 information literacy modules by interviewing faculty about assignment outcomes via survey. Usage data from LibWizard will provide additional insight. This is the first assessment of these library instruction modules. Our initial goals are that they are being used and that instructors find that they contribute to student learning.
This method will supply data for Learning Outcomes 1, 2, and 3 and Performance Outcome 4.
Method History:
This is a revised assessment method that has not been used before in its current form.
Results :
There were 34 sections of COM 1010 with 21 instructors and 1013 students enrolled. LibWizard reports indicate that 54 (5.33%) students completed Module 1 and 11 (1.09%) students completed Module 2. The other instruction modules are embedded in Canvas, so usage reports are not available. It is important to recognize that the library instruction modules are optional, and it is clear from the data that many instructors do not integrate them into their curricula.
Student scores for Module 1: 24 at 100%, 22 at 80%, and 8 at 60%
Student scores for Module 2: 5 at 100% and 6 at 60%
There were 6 (28.57%) responses to the faculty survey with only 4 of 6 (19.05%) stating that they used the modules. Further, only 2 (9.52%) surveys were completed. There is insufficient data to determine if the modules are being used and if they contribute to student learning.
Program Action Plan:
A team of librarians will evaluate this instruction initiative, including the recent data, to decide if it should be retained, redeveloped, or retired. If it were to continue, faculty participation is essential. Strategies will be outlined to encourage wider course integration.
Timeline for Action Plan Implementation:
A determination will be reached before the start of the Fall 2024 semester with sufficient time to create or update the instruction modules and implement them.
Interlibrary Loan Services
Assessment Method:
The University Libraries provide 24/7 access to our Interlibrary Loan (ILL) service to procure information resources from beyond the library collections for patrons (students, faculty, staff). ILL software captures transactional data such as the number of requests made, and the number of requests filled. This is a useful metric that we use to monitor our service performance. We strive to fill 85% or more of ILL requests. We also monitor the resource requests to inform collection development and library acquisitions. For this assessment cycle, in addition to measuring our fill rate, we want to better understand how this service is used by students. The Director of Library Resources and Collections Strategy Services will work with a small team of Librarians to gather and analyze ILL requests to determine the percentages of requests made by students, and then further analyze by discipline and class standing. We will then examine the fulfillment rates of student requests to determine if there are significant gaps in the collections. This is our first assessment of ILL Services. Data analyses and results will inform future goal setting in this area.
This method will supply data for Performance Outcome 11.
Method History:
This is a new assessment method that has not been used before in its current form.
Results :
For our first inquiry into ILL activity, as part of our annual assessment plan, we examined data for journal article requests for the period from January 1, 2019 - April 30, 2024 (16 semesters; 72,201 requests). We determined that 93% of article requests were filled, exceeding our minimum target goal of 85%.
Analysis of the aggregated requests from all patron types was useful to inform collection development strategies. Based on high numbers of ILL requests for content from select journals, the Libraries were able to identify and purchase backfiles for 38 journal titles to fill gaps in the collection and provide immediate access to articles.
Our intention was to further analyze the transaction logs to determine what percentage of the ILL requests are submitted by students and then determine their disciplines and class standings. Unfortunately, we learned that, due to self-reported demographics in the request form, there are significant inaccuracies that prevent us from completing this portion of our inquiry.
Program Action Plan:
We will increase our target goal for filled ILL requests to 93% and continue to analyze the data annually for one-year periods (3 semesters) from May- April to identify additional opportunities to purchase journal content to meet patron needs. Further, we will explore alternate means of identifying accurate demographic information to better understand the patron types that utilize ILL services.
Timeline for Action Plan Implementation:
This assessment method is an ongoing annual initiative. Data collection is in process. Additional demographic data is being explored in hopes of a viable solution for the May 2025 analysis.
We will share findings from this analysis with library administration and reference librarians during the spring/summer 2024 semester.
Stakeholder feedback for analysis was gathered from July 21 through August 18, 2024. Survey data was analyzed by a four-person working group, comprised of members of the Libraries Assessment Committee, during the week of August 25, 2024, to complete the cycle and feedback was incorporated into the University Libraries assessment website. The survey remains open to gather additional feedback during the fall 2024 semester. Stakeholders suggested additional areas of library services and resources for future assessments including Research Guides, library spaces, and library instruction sessions. Also suggested were:
Stakeholder feedback will be useful during development of the 2024/2025 assessment plan.
The Wayne State University Libraries advance scholarship, student learning and faculty innovation through continuous development of a library that serves as a national model for a research University with an urban teaching and service mission.
The Access Services Area advances student success by providing access to physical and digital collections from internal and external sources. The Access Services Area helps provide study space and technology resources that contribute to achievement in academic success for students.
The Library online environments and tools advance the academic success and personal growth of undergraduate, graduate, and professional students by providing them with a website containing information about library services and resources; tools for identifying information sources across all disciplines studied at the university; tools for retrieving digital books, articles, and other resources; and the means to request resources and services from the University Libraries.
Our public services librarians provide our undergraduate, graduate, and professional students with guidance and instruction, through its own initiatives and faculty collaborations, to support development of the information literacy skills needed to traverse a complex information ecosystem as student and career professionals.
The Reference/Research Consultation Area, through in-person and virtual interactions, advances the academic success and intellectual growth of undergraduate, graduate, and professional students through quality answers to their research questions and by offering guidance in developing their information-finding skills.
The Library Collections Area Libraries advances the academic success and personal growth of undergraduate, graduate, and professional students by providing them with information resources serials, journals, books, and media in physical and digital formats necessary for such advancement.
The Library Publishing Area promotes open scholarly communication through the production, delivery, and preservation of digital research projects, and provides undergraduate, graduate, and professional students with a platform for presenting their scholarly research.
The Walter P. Reuther Library collects and facilitates access to historical documentary evidence in order to inspire academic research, serve the information needs of the community, and build knowledge at Wayne State University.
University Library System Performance & Learning Outcomes (Updated May 2023)
Students will be able to locate books and journal articles on a given topic through tools (e.g., catalog, QuickSearch, databases) on the Libraries’ website.
First-year undergraduate students in FYS1010 will report an increase in confidence in applying strategies to evaluate information sources for reliability and relevance. In the evaluation following the completion of the Information and Media Literacy Module, 80% of the respondents who identified in Question 2 as being “Not knowledgeable at all” or “Slightly knowledgeable” about evaluating information they have found on the internet will identify as being more knowledgeable in Question 4 by selecting moderately, very or extremely.
Provide first-year undergraduate students in FYS 1010 with the instruction necessary to locate information for their assignments and studies. In the evaluation following the completion of the Information and Media Literacy Module, 75% of the respondents will identify in Question 3 at least 3 resources or services they will likely use to locate information sources for their papers and projects.
Provide instructors with the assistance necessary to compose “library assignments” that will encourage students to explore information resources and develop library research skills.
Provide reference services and research consultations to assist students with their projects and research needs.
Provide students with books that contribute to learning and growth.
Provide students with the streaming media that contribute to learning and growth.
Provide students with the Online Articles and Databases (digital journals/articles) that contribute to learning and growth.
Provide students with circulation services that contribute to learning and growth.
Provide students with reserve services that contribute to learning and growth.
Provide Interlibrary Loan services (articles, book chapters and books in print or PDF) that contribute to learning and growth.
Provide a website that students use easily and efficiently to meet their study and research needs.
Provide reservable study rooms that students find conducive to study and academic work.
The WSU Libraries stakeholders include: Students, Program colleagues (e.g., staff, librarians, administrators),WSU colleagues beyond the Libraries (e.g., non-Libraries faculty, staff, administrators), Non-WSU stakeholders (e.g., community or industry partners, advisory groups, employers, accrediting bodies).
The University Libraries maintains a website (this LibGuide) that describes the annual program assessment initiative. It includes pages for each year's assessment plan and results. This year, a survey component was added to facilitate stakeholder feedback. The Libraries’ website and an Academica announcement promoted the assessment plan, results, and feedback request to a range of internal and external stakeholders. Internal stakeholders were encouraged to participate via system-wide emails and at a meeting on July 25, 2024.