Competency N: Assessment
Use assessment measures to evaluate programs and services.
Introduction
I understand this competency as the discipline of finding out whether something works, and for whom, before deciding what to do next. Every program and service rests on assumptions: that people can find it, that they can use it, that it teaches what it claims to teach, and that it makes a difference afterward. Assessment tests those assumptions with evidence. The competency speaks of assessment measures in the plural, and I think that is deliberate. No single measure can answer every question, and a good evaluation chooses its measures to fit the questions that matter.
Evaluation of training and instruction has a well-known framework. Kirkpatrick and Kirkpatrick (2016) describe four levels of evaluation: reaction, which is how participants respond to the training; learning, which is the degree to which they acquire the intended knowledge and skills; behavior, which is the degree to which they apply what they learned on the job; and results, which are the outcomes the training was meant to produce. Each level is harder to measure than the one before it, and many programs stop at the first, asking participants whether they enjoyed a session and treating a positive answer as success. The framework is a reminder that satisfaction and learning are different things. A pre-test and post-test, for example, measure the second level, which a satisfaction survey cannot reach.
Services that people use through an interface, such as websites, catalogs, and virtual environments, call for a different set of measures. Nielsen (1994/2024) proposed ten heuristics for user interface design, including visibility of system status, match between the system and the real world, consistency and standards, and recognition rather than recall. In a heuristic evaluation, a reviewer inspects an interface against these principles and records where it falls short. The method is fast and inexpensive, which makes it a sensible first step, but it reflects the reviewer’s judgment and not users’ behavior. It is strongest when paired with methods that involve real users, such as usability tests or card sorts.
Accessibility is a measure in its own right, and it can be checked against a published standard. The Web Content Accessibility Guidelines 2.2 set testable success criteria, including a minimum contrast ratio of 4.5 to 1 between ordinary text and its background (World Wide Web Consortium [W3C], 2023). A service that fails such a criterion is failing some of its users, however well it serves the rest, and an evaluation that ignores accessibility has left part of the audience out.
This competency matters to me because assessment is built into my training and my work. Human factors is an evaluation discipline: it measures how people perform with a system and treats errors as data about the design. As a freelance contractor I write rubrics used to evaluate the output of AI systems, where I have learned that a criterion that two reviewers would score differently is not yet a criterion. The competency matters to the profession because libraries are asked to show what their services achieve, and because assessment is how a library learns what to change. An unassessed service may be working well, or failing quietly, and no one can tell which. The evidence below shows me evaluating an organization’s website with several measures, designing an assessment plan for a learning experience, and building evaluation into a design cycle from user research through testing.
Evidence
My preparation for this competency came from INFO 251, Web Usability, where I learned heuristic evaluation and card sorting; INFO 298, Special Study, where I worked with a UX student team in Summer 2025; INFO 287-19, Design for Teaching and Learning in Social VR, where I designed learning assessment for an immersive game; and the Google UX Design Certificate. [Confirm whether the certificate is complete, and add any other course that covered assessment.]
Evidence 1: iLRN website usability review (INFO 251)
In spring 2026 I evaluated how a newcomer to the Immersive Learning Research Network would find information about joining and volunteering on the organization’s website. I used three measures in sequence. I began with a stakeholder interview with a member of the organization’s leadership to learn what the site was meant to achieve. I then conducted a heuristic review of the site against Nielsen’s principles and checked its color contrast against the WCAG criteria. Finally, I ran an unmoderated hybrid card sort in UX Tweak with six participants from outside the organization, using 20 cards written in plain language to represent the site’s content. [Add the dates and the number and severity of the heuristic issues you found. Link the site.]
This project demonstrates the use of several assessment measures to evaluate a service, with each measure answering a different question. The interview defined the criteria: what the organization needed newcomers to be able to do. The heuristic review and contrast check showed where the site fell short of established standards. [Add one heuristic finding and one contrast finding, with the heuristic it violated or the ratio you measured.] The card sort showed how real newcomers expected the content to be organized. Its clearest finding was that the cards describing what volunteering involves and the guide for new volunteers were each placed in the Volunteer category by five of six participants, while the site itself filed volunteer information under a menu labeled “More.” My recommendations followed directly from the evidence. [List your two or three main recommendations.] The project also taught me to report findings no more strongly than the data allow. With six participants, a placement shared by two people is a hint, not a pattern, and I revised my own report to say so. [Keep this sentence once you have made those corrections.]
Evidence 2: STEMvival: Winter assessment plan and test plan (INFO 287-19)
For STEMvival: Winter, an immersive winter survival scenario I designed to teach applied science and math to middle and high school students, I wrote an assessment plan and a separate test plan. The assessment plan sets two goals, practical STEM learning and increased self-efficacy toward STEM, and measures both. [Add the term. Attach both plans.]
The plan demonstrates assessment designed into a program from the start rather than added at the end, with a different measure for each question I needed to answer. A pre-test and a post-test delivered through Canvas measured content knowledge of winter survival, navigation, electronics troubleshooting, and heat transfer, which in Kirkpatrick and Kirkpatrick’s (2016) terms is the second level, learning. The pre-test was mandatory before learners could reach the game, and correct answers on the post-test revealed a full explanation, so the test reinforced learning as well as measuring it. Likert-scale questions on both tests measured self-efficacy and attitudes toward STEM, the first-level reaction measures were “fun” ratings and written reflections, and an indirect review by a focus group of independent instructors and subject experts was planned to judge whether the design met its goals. The plan also accounted for its learners. Achievements earned in the game added bonus points to the post-test, but the joking badges for mistakes carried no penalty, so that students who learn best by doing could show what they knew without being marked down for how they got there. The test plan then set out how the experience itself would be evaluated before release: observed playthroughs in which users narrate what they are doing, A/B tests of key interface elements, analytics on where users abandon tasks, testing across phones, tablets, computers, and headsets, and accessibility testing with users who have visual, auditory, motor, and cognitive disabilities. Together the two plans separate the question of whether students learned from the question of whether the design worked. [If any part was carried out, add what you found. Otherwise say that these were plans for the experience, which is now offline because Spatial.io no longer hosts it.]
Evidence 3: INFO 298 UX internship: research protocol, reflection, and Horse Sense Academy personas, journey maps, and wireframes
In Summer 2025 I worked with a UX student team on a web application being built to help SJSU students with degree completion, professional development, and portfolio building, and on a university web page for a book drive for Ukrainian children. The team followed a research protocol that moves from analytics and a literature review, through stakeholder interviews, focus groups, and surveys, to personas, critical task pathways, cognitive walkthroughs and task-based usability tests with five representative users, and finally to wireframes and further rounds of testing. I helped prepare the survey that invited beta testers for pre-release usability testing, took part in think-aloud usability testing, and wrote a reflection on the experience. During the same internship I produced my own set of personas, journey maps, wireframes, and virtual reality scene designs for Horse Sense Academy, a fictional online and VR course in horsemanship that I invented as a design exercise. The personas are fictional composites, and their portraits are AI-generated. The exercise later inspired my action research project with an equine-assisted services program, described in Competency L. [Confirm your part in the web application work, and attach the reflection and the Horse Sense deck.]
This work demonstrates assessment as part of the design cycle, not something added at the end. The protocol builds evaluation into every stage. The early methods assess what users need before anything is built, the walkthroughs and task-based tests assess whether a design meets those needs, and the iteration stage repeats the cycle. Of these, the think-aloud test was the method I valued most. Asking a user to say what they are thinking while they work reveals the reasoning behind each error, which a completion rate or a satisfaction score cannot show. In my reflection I noted that I want to use it to test the things I make in the future.
The Horse Sense deck shows me designing assessment into a product from the needs of its users. I wrote two personas. One is a recent forestry graduate who must learn to handle horses within three months to qualify for a backcountry ranger job, and whose pain points include having no way to test whether he is learning correctly. The other runs a therapeutic riding center and needs volunteers trained before they arrive, and her central question is how she will know they have learned what they need. Both personas describe an assessment problem, so the designs answer it at the level Kirkpatrick and Kirkpatrick (2016) call learning. The journey maps include a placement quiz that customizes each learner’s path and monthly progress reports for the riding center’s owner. The wireframes add a dashboard with progress bars, learning pathways, and achievements. The VR scenes check knowledge through action: the learner taps a part of the horse to learn its name, selects the correct tool from the tack room, and names each grooming tool aloud before using it. [Add whether either design was tested with users, or say that testing was the planned next step.] My reflection closed with a principle I still hold for evaluation: a design that looks finished should be checked against accessibility features such as alternative text and screen reader support, because a design is only as good as its experience for the users at the margins.
Conclusion
I will apply this competency by deciding at the start of any program what success would look like and how I will measure it, by using more than one measure whenever the stakes justify it, and by reporting results no more strongly than the evidence allows. My next application is the equine-assisted services project, where I will compare newcomers’ familiarity and comfort before and after using the virtual environment and the take-home kit. I expect to use the same skills in library assessment, instructional design, and AI evaluation. To stay current I will follow Nielsen Norman Group, the W3C Web Accessibility Initiative, and the Association of Research Libraries’ Library Assessment Conference. [Add any assessment resource you use.]
References
Kirkpatrick, J. D., & Kirkpatrick, W. K. (2016). Kirkpatrick’s four levels of training evaluation. ATD Press.
Nielsen, J. (2024, January 30). 10 usability heuristics for user interface design. Nielsen Norman Group. https://www.nngroup.com/articles/ten-usability-heuristics/ (Original work published 1994) World Wide Web Consortium. (2023). Web content accessibility guidelines (WCAG) 2.2. https://www.w3.org/TR/WCAG22