Introduction

If information organizations do not evaluate their programs and services, there is no way to know how they can be improved. In a society where “times are changing and the very existence of physical libraries is being questioned,” evaluation is more important than ever to ensure services are efficient and cost-effective (Cassell & Hiremath, 2023, p. 395). Without assessing how the organization is working, it cannot be improved upon. Evaluations must use measurable criteria, or there would be no way of knowing how programs are performing. This is similar to a creating a “SMART” goal: a goal that is specific, measurable, achievable, relevant, and time-bound.

Assessment

Over the past few decades, assessment has become more common in information organizations. Ryan (2006) defines assessment (in a library setting specifically) as “any activities that seek to measure the library’s impact on teaching, learning and research as well as initiatives that seek to identify user needs … with the overall goal being the data-based and user-centered continuous improvement of our collections and our services.” Assessment can be thought of as a cycle: developing assessment questions or objectives, identifying assessment methods and data gathering, analyzing and interpreting data in terms of the assessment question or objective, integrating results into operational and strategic planning, and disseminating results — then the cycle can continue from the beginning (Bakkalbasi, 2017, p. 215). Assessment is an ongoing process.

According to Cassell & Hiremath (2023), the most used methods of assessment are, in order of simplest to most complex, suggestion boxes, surveys, interviews, observations, focus groups, case studies, and user experience (p. 397). Quantifiable data can make a big difference when seeking external support or funding. Since this can be time consuming and expensive, an information organization might want to consider buying a toolkit like LibQUAL, a library user survey tool which can “identify best practices, analyze deficits, and effectively allocate resources” (LibQUAL, n.d.).

Measurable Criteria

Outcomes need to be measurable, otherwise there is no way to know if progress is being made. Rubin (2006) defines an indicator (another term for criteria in this instance) as “a statistic that summarizes the user’s achievement of an outcome” (p. 33). More specifically, an indicator “has a verb, an object for the verb, a quantity, and a time frame” (Rubin, 2006, p. 34). For example, if a desired outcome for a program is “participants learn of new job or career opportunities,” its indicator could be, “participant lists four more job options after the program than before the program” (p. 34).

Some organizations have standardized guidelines or criteria that can be used by information organizations. The Reference and User Services Association (RUSA) offers guidelines for the behavioral performance of “library workers in reference and information services, in all types of libraries, who answer informational questions or conduct reference interviews” (RUSA, 2023). An example guideline is approachability, which is important to maintaining a welcoming atmosphere. An approachable library worker meets criteria like “can be easily identified as a staff member” and “avoids jargon and clarifies confusing terminology.” The International Organization for Standardization (ISO) publishes standards for a wide range of sectors, and one of their standards, ISO 11620:2023, “Information and documentation — Library performance indicators,” is designed for assessing libraries of all kinds.

Evidence

Artifact 1

Assignment: Comparative Analysis of Major Subscription Databases: JSTOR and Project MUSE
Course: INFO 210, Reference and Information Services
Description: This was a group assignment that tasked us with comparing two major databases, JSTOR and Project MUSE. We were to analyze the databases from the point of view of a small library with limited financial resources, so a careful examination of what to spend money on would be necessary. We were given specific criteria to analyze the databases: scope, quality of content, accuracy, currency, authority, ease of use, arrangement, and appropriateness.

This assignment demonstrates my ability to evaluate services using criteria. In addition to some of the overall writing and editing of the report, my main contribution to this assignment was the section about the ease of use criteria. According to Cassell & Hiremath (2018), “the most relevant factors [for ease of use] are whether the program is easy to search and how quickly it responds to commands or queries,” as well as if it allows Boolean operators, has basic and advanced search capabilities, whether it is easy to browse, and whether it has “Help” or “How to” sections (p. 352). Through testing, I found that JSTOR had a better support page, allowed Boolean operators in text searches, had a Help button, and loaded search results faster. Thus, I was able to determine that JSTOR was the better option for our library based on ease of use.

Artifact 2

Assignment: Unit 2 Discussion Post
Course: INFO 210, Reference and Information Services
Description: One part of this three-part discussion assignment asked me to perform a content evaluation for a website of my choosing using provided criteria. The criteria fell into the categories of first look, information providers, information currency, information quality, and further information. At the end, I had to tally up whether each criteria applied and rate the website’s usefulness overall.

I chose to evaluate a local art museum’s website. I found that the website was overall very useful because it met criteria like having contact information, being updated frequently, linking to other currently working sites, being well organized, and being free of bias. Using the evaluation criteria also highlighted the website’s weaknesses, a lack of information about pages’ authors and no revision dates. Since those weaknesses were minor and few in number compared to the number of criteria it met.

Artifact 3

Assignment: Reference and Information Services Interactions
Course: INFO 210, Reference and Information Services
Description: For this assignment, I needed to prepare two questions to ask reference librarians at one virtual and one in-person library. I used the RUSA guidelines to analyze the reference interviews, focusing on the criteria of approachability, interest, listening, searching, and follow-up (Cassell & Hiremath, 2018, p. 25-26). I also used the hidden observation method, meaning I acted as a patron asking a reference question and not an MLIS student conducting an evaluation. I used the unobtrusive observation questions worksheet to guide my thoughts as well (p. 405).

This demonstrates my ability to evaluate reference services based on established guidelines. Both experiences were overall positive for me, even though there were some actions I wish the librarians had taken. I was able to connect both visits to the guidelines and questions from the text while having successful reference experiences. The criteria helped guide my understanding of what to look for.

Conclusion

In the future, I plan to utilize these skills to evaluate how my information organization is performing. I will remain current by keeping up with organizations like RUSA that publish guidelines, as well as professional development like the American Library Association’s webinars on evaluating services.

References

Bakkalbasi, N. (2017). Assessment and evaluation, promotion, and marketing of academic library services. In T. Gilman (Ed.), Academic Librarianship Today (pp. 211-221). Rowman & Littlefield.

Cassell, K. A., & Hiremath, U. (2018). Reference and information services: An introduction (4th ed.). ALA Neal-Schuman.

Cassell, K. A., & Hiremath, U. (2023). Reference and information services: An introduction (5th ed.). ALA Neal-Schuman.

LibQUAL. (n.d.). Survey FAQs. https://www.libqual.org/about/faq_survey

Reference and User Services Association [RUSA]. (2023). Guidelines for behavioral performance of reference and information service providers. American Library Association. https://www.ala.org/rusa/resources/guidelines/guidelinesbehavioral

Rubin, R. J. (2006). Demonstrating results: Using outcome measurement in your library. American Library Association.

Ryan, P. (2006). About. Library Assessment. https://web.archive.org/web/20170920081259/http://libraryassessment.info/?page_id=7