Capturing Outcomes of Youth Computational Thinking Programs in Public Libraries: A Repository of Assessment Tools and Case Studies of Impact

The concept of computational thinking (CT) and its importance to all young people was widely popularized by Jeannette Wing in her influential 2006 article. That article states that “Computational thinking is a fundamental skill for everyone, not just the computer scientists” (Wing, 2006, p. 3) going on to say that CT should be added along with reading, writing, and arithmetic as an essential skill for all students. She defined CT as “…solving problems, designing systems, and understanding human behavior, by drawing on concepts fundamental to computer science” (Wing, 2006, p. 13). While there is little consensus as to a single definition of CT, it is generally agreed upon that CT practices include problem decomposition, developing and using abstractions, debugging, defining algorithms, and concepts grounded in the practice of programming such as iteration, parallelization, and conditional logic (Grover & Pea, 2013). Our operationalization of CT, which we will refer to as CT literacy, adheres closely to the structure and content presented by Grover & Pea and includes both practices and concepts from the discipline of computer science.
In a recent consensus study report published by the National Academies of Science, Engineering, and Medicine, public libraries were identified as a crucial setting in an ecosystem of learning experiences that is essential in the building of interest, knowledge, and competencies for computing for groups that are underrepresented in computing because of their gender, race, ethnicity, and perceived ability (NASEM, 2021). Public libraries in particular are uniquely positioned to help with this issue due to their ubiquity and the fact that parents and children recognize libraries as “community institutions'' (Braun & Visser, 2017, p. 9) that provide a myriad of learning opportunities, an important, but often-overlooked dimension of broadening participation (DiSalvo, Reid & Roshan, 2014). Braun & Visser (2017) conducted a nationwide survey of libraries offering CT or computer science (CS) programs for youth and found that libraries offer such programs in abundance. However, very little empirical evidence has been produced that can point to the actual CT literacy practices and outcomes that libraries are well positioned to develop in young adults.
Based on a three-year IMLS-funded project conducted by researchers at the University of Maryland, we will present the learning outcomes that can be achieved through CT programs for youth ages 11-18 offered through public libraries. We will also share a suite of assessment tools designed for public library staff to document and measure CT literacy development in youth as a result of participating in CT programs through their libraries. The learning outcomes were identified through nationwide interviews and focus groups with 59 library staff. We identified seven CT outcomes that can be achieved through CT programs in public libraries, which are:

(1) Positive CT Experiences - The program provides youth with a positive experience with CT and excites them to pursue future CT experiences.
Preparation for Future Learning / Career Prep - The program prepares youth for STEM and CT coursework in school, and/or introduces youth to STEM and CT careers.
(2) CT Conceptual Learning - The program helps youth acquire conceptual knowledge associated with CT, which includes designing and planning, prototyping, user testing, modifying and revising, testing, debugging and troubleshooting, and documenting a CT project.
(3) CT Dispositions - The program improves youths’ dispositions towards CT, which includes their tolerance for ambiguity, persistence, and the ability to work collaboratively with others.
(4) CT Skills - The program helps youth develop CT skills such as problem decomposition, iterative development, and debugging.
(5) CT Confidence - The program instills a sense of “I can do this” in youth when they are involved in computational-related projects and digital technologies more broadly.
(6) Access and Equity - The program introduces youth to new technologies and “evens the playing field” for those who might not have access to new technologies and CT education elsewhere.

Based on these empirically grounded CT outcomes, we designed assessment tools to measure these outcomes. We worked with 12 library staff nationwide through a series of participatory design sessions (Druin, 1999, 2005) whereby we engaged with library staff to obtain external feedback on the format and content of the assessment tools and iteratively developed prototypes of these tools. Then, we worked with 19 libraries that were offering CT programs for youth ages 11-18 to pilot our tools. In this presentation, we share the resulting seven types of CT assessment tools that are customizable and can be decomposed into smaller tools to fit the needs of public library staff. The tool groups are:
(1) Observational Checklist - to measure CT dispositions.
(2) Talkback Boards - to measure a variety of outcomes.
(3) Design Scenarios - to measure CT Skills.
(4) Rubric - to measure CT conceptual learning.
(5) "Build Your Own" Interview Protocol - to measure a variety of outcomes.
(6) Survey - to measure CT confidence.
(7) Facilitator Self-Assessment - to measure access and equity.

As part of the process of piloting the assessment items, the library staff from the 19 libraries that worked with us shared the data they collected with the assessment tools as well as their experiences using these tools. In this presentation, we will share a repository of case studies of how public libraries used these assessment tools and shared the impact of their program with their community, as well as using what they have learned to improve the design and implementation of their CT programs. These case studies include CT programs that range from physical computing/robotics (e.g., Sphero, e-textiles), coding (e.g, Scratch, web development, Python/Javascript courses), makerspaces and digital fabrication (e.g., 3D printing), multimedia production (e.g., sound engineer), and unplugged offline CT activities, for young library patrons (ages 11-18).

References:
Braun, L., & Visser, M. (2017). Ready to Code: Connecting youth to CS opportunity through libraries. Washington, DC: American Library Association.
DiSalvo, B., Reid, C., & Roshan, P. K. (2014). “They can’t find us: The search for informal CS education.” SIGCSE 14: Proceedings of the 45th ACM Technical Symposium on Computer Science Education. 487-492.
Druin, A. (2005). What children can teach us: Developing digital libraries for children with children. The Library Quarterly, 75(1), 20-41. https://doi.org/10.1086/428691
Druin, A. (1999, May). Cooperative inquiry: developing new technologies for children with children. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (pp. 592-599). https://doi.org/10.1145/302979.303166
Grover, S. & Pea, R. (2013). Computational thinking in K–12: A review of the state of the field. Educational Researcher, 42(1), 38–43. https://doi.org/10.3102/0013189X12463051
National Academies of Sciences, Engineering, and Medicine. (2021). Cultivating interest and competencies in computing: Authentic experiences and design factors.Washington, DC: The National Academies Press. https://doi.org/10.17226/25912.
Wing, J. M. (2006). Computational thinking. Communications of the ACM, 45(3), 33–35.