Competency E: Information Retrieval Systems
Design, query, and evaluate information retrieval systems.
Introduction to Competency E
I understand this competency as three linked abilities. To design an information retrieval system is to decide what will count as a record, which attributes of each record will be captured, how those attributes will be entered, and how a user will search them. To query a system is to turn a need into a search the system can act on. To evaluate a system is to judge whether people can find what they are looking for with reasonable effort. The three depend on each other, because a design can only be judged by querying it, and every flaw found in evaluation leads back to a design decision.
Chowdhury (2010) introduces information retrieval as the work of connecting a collection of information to the people who need it. Documents are represented by index terms or field values, queries are represented in the same vocabulary, and the system matches one against the other. This means the designer’s choices about representation set the limits of what a user can ever retrieve. If a property is not recorded, nobody can search by it. If it is recorded inconsistently, searches will miss records that should match.
Evaluation has a standard vocabulary. Manning et al. (2008) define precision as the share of retrieved items that are relevant and recall as the share of relevant items that are retrieved. The two usually pull against each other. A system that returns everything has perfect recall and poor precision. A system that requires many exact conditions has high precision and may return nothing at all. In my coursework I came to think of design as the work of placing a system at the right point between those two failures for a particular group of users.
Retrieval is also broader than the search box. Rosenfeld et al. (2015) describe the information architecture of a website as a combination of organization systems, labeling systems, navigation systems, and search systems. A person who clicks through a menu is retrieving information just as much as a person who types a query, and a confusing label is a retrieval failure in the same way a bad index term is.
This competency matters to me because of my work with AI systems. Many current AI products depend on retrieval: a system finds documents related to a question and passes them to a language model to compose an answer. When the retrieval step returns the wrong documents, the answer is wrong, however fluent it is. My evaluation work often comes down to the same questions I learned in INFO 202. What was the system able to match on? What did it miss, and why? Understanding fields, vocabularies, precision, and recall lets me explain a failure and not only report it.
The competency matters to the profession because retrieval systems are how people now meet nearly all information, and most of those systems are designed by people with no training in how users search. Information professionals bring two things that are often missing: attention to the user’s own language and a habit of testing. A catalog, a database, a website, and a chatbot are all retrieval systems, and all of them can be designed carelessly. The evidence below shows me designing a system, evaluating that design, and building the vocabulary a system depends on.
Evidence
My preparation for this competency came from INFO 202, Information Retrieval System Design, which I completed in Spring 2025. The course required a group to design and build a working database and then to evaluate it. All three pieces of evidence come from that course, and the first and third are group work. [For each group piece, the portfolio requires a clear statement of your role. I have stated what your reflection says; add anything missing.]
Evidence 1: Project 1: Beta Prototype, Canned Coffee Drinks (database design document)
This nine-page document, dated March 7, 2025, specifies a searchable database of ready-to-drink coffee products for young adult consumers. It was produced by a group of four. I wrote the Statement of Purpose and researched the rules for the Fair Trade Certified field. The document contains the Statement of Purpose, screenshots of the search page and the submission form, and a rule for each of the nine fields.
The document demonstrates design. The Statement of Purpose defines the users, adults aged 18 to 30, and derives the fields from what those users want to know: flavor, caffeine level, sugar and dairy content, price, size, and fair trade status. It gives sample searches, such as a sugar-free, dairy-free coffee with moderate caffeine. Each field rule states the field type, whether it is required, a definition, and an instruction for the person entering data. The rules anticipate inconsistency. Caffeine is entered as low, moderate, or high with numeric limits for each level, and the rule says which level a product on the boundary belongs to. The sugar-free field follows the federal labeling threshold of less than half a gram per serving. For fair trade status we limited the field to certification by three named organizations, because my research showed that fair trade claims on packaging are loosely regulated and third-party certification is more trustworthy.
Evidence 2: Project 1 Reflection (individual evaluation)
This is my individual reflection on the same project, written after the database was built and tested.
The reflection demonstrates evaluation. I identified the design’s main weakness as too many required fields. Caffeine level, sugar free, and dairy free were all required, so a user could not browse by caffeine range alone without also making choices about sugar and dairy. In the terms of this competency, the design favored precision so strongly that it damaged recall. I also described edits the group made after feedback to improve aggregation and discrimination in the numeric fields, by removing overlap between ranges. My conclusion was that the designers of a database hold considerable power over what users are able to find, and that the central task is balancing specificity against a useful number of results. Reviewing the design document again for this portfolio, I can see further problems of the same kind. The size ranges leave a gap, so that a five-ounce product fits no category, and the fair trade rule mixes instructions for the person entering data with instructions for the person searching. [Keep the last two sentences only if you agree with them after checking the document.]
Evidence 3: Exercise 2: Vocabulary Design Basics (controlled vocabulary worksheets)
In this group exercise we built a small controlled vocabulary and used it to index bibliographic records. We identified the main concepts in four records, grouped similar concepts, chose a preferred term for each group, and then assigned three or four descriptors to each record from the final list. [State your role in the group.]
The worksheets demonstrate the representation step that every retrieval system depends on. They also show the cost of a vocabulary that is too small. Our final list had fewer than ten terms. When we indexed the records, the descriptor Information Access was assigned to three of the four records, and so was Library Access Tools. A term that applies to nearly every record does not help a searcher tell records apart. An article on back-of-book indexing received the descriptor Reference Services, which fits it only loosely, because nothing more specific was available. The exercise taught me that vocabulary design is a retrieval decision. Broad terms raise recall and lower precision, and the right level of specificity depends on how large the collection is and what distinctions its users need to make. [This exercise is also used in Competency G. Present it there as an example of vocabulary construction.]
Conclusion
I will apply this competency in my work evaluating AI systems that depend on retrieval and in any role where I design a database, a catalog, or a website. I will begin with the users and their questions, record only what they will search by, and test the system with real queries before calling it finished. To stay current I will follow the Journal of the Association for Information Science and Technology, the proceedings of the ACM SIGIR conference for research on retrieval and its evaluation, and the Nielsen Norman Group for findings on how people search and navigate. [Add any source you read for your AI evaluation work.]
References
Chowdhury, G. G. (2010). Introduction to modern information retrieval (3rd ed.). Facet Publishing.
Manning, C. D., Raghavan, P., & Schütze, H. (2008). Introduction to information retrieval. Cambridge University Press.
Rosenfeld, L., Morville, P., & Arango, J. (2015). Information architecture: For the web and beyond (4th ed.). O’Reilly Media.