As the field of higher education contends with “the great resignation” and institutions work to combat burnout, the demand for tools and support for practitioners to advance equity-conscious practices has never been more tangible. The identification of high-quality programming as a strategy to advance equitable outcomes across Illinois’ higher education systems, particularly for Black, Latinx, low-income, rural, and working adult populations (Illinois Board of Higher Education, 2021) precipitates the need for devices that empower comprehensive assessment. In continued efforts to support practitioners and the development of equitable programming and outcomes for students, OCCRL engaged the field in participatory action research toward the development of an equity-centered rubric to advance learning and support for the Illinois CTE Program Review Process. The Equity Centered Program Review Rubric for Career Technical Education is an institutional assessment tool that provides a framework for institutional improvement efforts and fosters transformational change. As an instrument that considers equitable approaches, the rubric encourages active stakeholder participation in the review process to propel throughout assessment discussion and critical thought about the program review as both, a process and deliverable. The rubric builds on the four goals of the initial evaluation study of the program review process: 1. “improve the efficiency and efficacy of the program review process by identifying challenges, redundancies, and omissions, and provide recommendations for refining the process. 2. Examine variations of the program review process across institutional contexts and institutional identities (e.g., rural/suburban/urban, minority-serving institutions/ predominately white institutions, small/large student populations) to understand how the process is utilized across diverse institutions throughout Illinois. 3. Identify professional development, technical support, and supplemental materials that could improve outcomes associated with the program review. 4. Enhance the application of program review findings in colleges’ campus-level programmatic planning and decision-making” (Fox, Thrill, & Keist, 2018, p. 1) In direct response to the third goal of the evaluation study is to create relevant supplemental materials to enhance the outcomes of the program review process. In direct response, to the third goal of the evaluation study, OCCRL’s rubric is positioned as relevant supplemental material meant to enhance the outcomes of the program review process. The rubric attends to the findings of an initial evaluation study where participants identified that job aids such as rubrics could be used to support internal feedback and professional development as well as structure feedback from the Illinois Community College Board (ICCB). Participants summarized… ‘that was basically for the use with the ICCB WHAT’S IN YOUR TOOLKIT? Utilizing Rubrics to Support the Program Review Process By Jewel A. Bourne UPDATE - WINTER 2023
feedback, perhaps they could establish a rubric that would be shared with all the colleges, so that you would know some items and what the various criteria would be for “emerging, meeting, and exceeding” on that rubric.’” (Fox, Thrill, & Keist, pp. 21-22). In alignment with the definition of program review a type of evaluation of institutional, instructional support, student service or administrative program, department, or unit (Beers, 2011, p.63-64), the rubric serves to support practitioners as they engage in the review process. The rubric furnishes benchmarking as a best practice by which to assess their response and document areas of strength and opportunities for growth. In alignment with the goals of statewide program review as provided by ICCB: The rubric advances equity-guided program review for Career Technical Programs in Illinois by cultivating environments that • concentrate on strategic campus-level planning and decision-making related to instructional programming and academic support services, including but not limited to program revisions, program closures, and revisions to services. • address inequities to support program improvement. • support the delivery of locally responsive, cost-effective, high-quality programs and services across Illinois’ community college system. (ICCB, 2020) As one of the stated purposes of program review is to address inequities (the use of the rubric enables institutions to leverage program review to attend to equity gaps and engage in ongoing, consistent, and specific analysis to evaluate progress. While the rubric focused on the quality of response, it also foregrounds the development of inquiry that is contextualized and intentional. The tool is attentive to the different processes across community colleges’ distinct geographical locations and unique organizational structures and resources. This design serves to encourage the prioritization and inclusion of data indicators that concentrate on equity gaps. These approaches are beneficial in adapting a campus culture that views program review as a valuable process. Utilization of the program review rubric can shift faculty and staff perspectives of the process from laborious and compliancebased to a more meaningful data analysis that enacts program improvement and contributes to a culture of transparency. The rubric can be activated as a tool to gauge the extent of faculty and administrative participation. Furthermore, it can be engaged as a mechanism by which to evaluate areas where all stakeholders of the campus, including academic support professionals and students, are currently involved and forecast areas in which they can be consulted. As it relates to stakeholder involvement, consideration of the data-driven rubric submissions provides opportunities to evaluate the impact of programmatic elements resulting from the program review. The design elements of the rubric facilitate the identification of challenging areas. Receptivity of these areas for growth, in turn, promotes the implementation of feasible, measurable and actionable steps. Furthermore, the rubric’s promotion of consistent data analysis advances opportunities to evaluate the immediate impact of changes that result from the program review process. From this practice, new questions are potentially stimulated surrounding program improvement, including the production of new areas for analysis, including disaggregated data needs that address equity gaps in programming, performance, and service. As a benchmarking tool, the rubric provides an opportunity to examine the extent to which the program satisfies elements of the template (need, cost, quality, incorporation of student voice, overall review, and process of review). The rubric affords contextualized program evaluation of responsiveness to local needs, cost efficiency, and integration of quality programming components. The rubric supports the development of a program review process that is well documented. Additionally, the use of the program review rubric contributes to the implementation of the review process to inform campus planning initiatives, quality improvement efforts, and budget allocation decisions as it requires institutions to consider their responsiveness to advancing equity. As we consider the urgency of resources that equip institutions and practitioners to advance and embed racial equity and favorable outcomes for historically minoritized groups, the rubric advances an environmental shift regarding the program review process as “one more thing to do, as something that we check a box on, get it done and sits on a shelf and is never again looked at until the next time you are scrambling to complete it.” (Georges, Jr. & Bourne, 2021,11:15) to a continuous improvement model that informs overall strategic planning and actionable items. Furthermore, the utilization of the rubric to probe programmatic methods and outcomes demonstrates a commitment shared throughout the college geared toward UPDATE - WINTER 2023
fostering systemic and localized changes that improve equity for underserved students (Fox, Thrill, & Keist, p. 8).” While the statewide program review process for Illinois community colleges required by the Illinois Community College Board (ICCB) is scheduled on a five-year cycle, it is also a measure that contributes to a continuous process that speaks to the fact that “the value of program review is lost without regular opportunities for engagement and reflection on what has been learned through the process” (Eggleston, 2020). Program review benefits institutions as an addition to their evaluation toolkit that contributes to the discussion of program strengths and arranges areas for improvement by prioritization of data identification and analysis. The rubric provides an opportunity to measure improvements and ensure incorporation as a part of the college’s strategic planning. Furthermore, the rubric affords the exploration of mechanisms and methods to increase institutional buy-in across administration and faculty. The Equity Centered Rubric for CTE Program Review in Illinois Community Colleges (OCCRL, 2022) is a tool pursuant to the completion and submission of the program review and provides opportunities for programs to explore opportunities for advancement congruent with the ICCB goals for Career Technical Education to participate in the reduction of inequities, strengthen programming and contributing to the economic development of the state. (ICCB, 2021) References Fox, H. L., Thrill, C. R., & Keist, J. (2018). Advancing program review: Evaluating and envisioning the future of program review at Illinois community colleges. Champaign, IL: Office of Community College Research and Leadership, University of Illinois at Urbana-Champaign. Georges, Jr., C.T., & Bourne, J. A. (2021, January 28). Conversations with practitioners: The importance of quality program review to advance community college student support services program review [Webinar]. University of Illinois at Urbana-Champaign Office of Community College Research and Leadership; Illinois Community College Student Activities Association. Illinois Board of Higher Education. (2021). A thriving Illinois: Higher education paths to equity, sustainability, and growth. Illinois Community College Board (2020). Program Review. Illinois Community College Board. (2021). Program Review Manual: Fiscal Year 2022-2026. Office of Community College Leadership (OCCRL). (2022). Equity-Centered Rubric for CTE Program Review in Illinois Community Colleges. “Manuscript in preparation”. University of Illinois at Urbana-Champaign. UPDATE - WINTER 2023 Jewel A. Bourne can be reached at [email protected].