CAEP Accountability Measures

The Bellarmine University Annsley Frazier Thornton School of Education (AFTSOE) prioritizes the use of data as part of its assessment and continuous improvement process. The data provided below represents a snapshot summary of survey reports, state and school district data reports, and teacher candidate performance assessment data. Data are acquired from sources including Jefferson County Public Schools (JCPS); the Kentucky Education Professional Standards Board (EPSB); Educational Testing Service (ETS); Bellarmine Career Development Center and the Bellarmine University Office of Institutional Research and Effectiveness. We have provided links to these data to assist prospective and current students, faculty, accrediting bodies, the public, and researchers.

The following programs were reviewed and approved by CAEP during our Fall 2019 accreditation visit:

  • Bachelor of Arts (B.A.) in Education
  • Master of Arts in Teaching (MAT)
  • Master of Arts in Education (MAEd)
  • Doctor of Education in K-12 District Leadership (Ed.D.)
  • Educational Specialist degree in Instructional Leadership and School Administration (Ed.S.)
  • Rank I individualized program in Curriculum and Instruction

Measures of Program Impact

Measure 1: (Initial) Completer Effectiveness and Impact on P-12 Learning and Development (CAEP Component R4.1)

Measure 1 (Initial): Completer Effectiveness (R4.1) 

The greatest number of our completers obtain work in the Jefferson County Public School (JCPS) system. To understand the teaching effectiveness of our 1st-3rd year completers, we examine data from various sources within JCPS. To surmise the impact our 1st-3rd year teachers have on P-12 learning and development, we examine student performance in reading and math on the annual Kentucky Summative Assessment (KSA). To surmise our 1st-3rd year completers effectiveness in applying professional knowledge, skills, and dispositions, we examine the results of a statewide survey: the Kentucky Impact Survey at district and school levels. The Kentucky Impact Survey is administered every other year, creating a challenge when comparing data across cycles.

Contribution to K-12 Student Learning Growth 

To infer completer impact on K-12 student learning, we compare data reflecting the performance of students taught by our 1st-3rd year completers with the growth of early career teachers in the district. Because Bellarmine sample sizes do not meet threshold for science and social studies, we limited to reviewing data in reading and math. For all three cycles (2021-2022, 2022-2023, and 2023-2024), the district provided scale scores on the Kentucky State Assessment in reading and math.

In cycle 1 (2021-2022) and cycle 2 (2022-2023), our completers performed at similar levels to other early career teachers. Cycle 1 (2021-2022) data indicate one to one to two points of difference in the means, particularly in elementary reading and math and high school math. In this same cycle, Bellarmine teachers performed between three and five points better in middle and high school reading and just under three points below in middle school math. Cycle 2 (2022-2023) means indicate similar performance in elementary reading, middle school reading, and high school math, with differences of less than two points. Bellarmine completers performed slightly better than other early career teachers in elementary math and high school reading with differences of less than four points, and slightly worse in middle school math, with a difference just under five points.

Cycle 3 (2023-2024) data present as distinct from the previous cycles with Bellarmine means falling notably below district means in elementary math and reading and middle school math. These differences range from eight to 15 points. Bellarmine completers perform notably above district means in middle and high school reading, by 30 and 23 points respectively. Means in high school math fit the pattern of more similar scores with a just under a four-point difference between Bellarmine completers and other early career teachers. In terms of directionality across all three cycles, Bellarmine completers outperform early career peers in high school reading and underperform in middle school math. Cycle 3 data indicate that completers at the elementary level struggled in 2023-2024, a distinction for this cycle. The span of difference in cycle three is something we will continue to monitor.

Completer Effectiveness in Applying Professional Knowledge, Skills, and Dispositions 

We examined the Kentucky Impact Survey for two components: Educating all Students and Managing Student Behavior. This survey is administered every two years, reflecting completer effectiveness for cycle 1 (2021-2022) and cycle 3 (2023-2024). There were no data available for cycle 2 (2022-2023). Data below reflect comparisons between district level data and elementary school data where our early career educators teach. For cycle 1, we were able to identify two schools where three or more of our teachers work. For cycle 3, we were able to identify two schools where six or more of our teachers work. These schools present cases through which we can consider the effectiveness of our early completers as compared to early career teachers across the district.

Cycle 1 (2021-2022) and cycle 3 (2023-2024) data provide a partial snapshot of effectiveness. At schools with notable numbers of Bellarmine completers, our completers perform consistently with early career teachers across the district when it comes to educating all students. Specifically, our completers rate themselves highly when it comes to using assessment to inform instruction and approximate district averages when it comes to including material about people from different backgrounds in the curriculum. Managing student behavior is an area for growth across both cycles 1 and 3. While these data are generally low, indicating a challenge for most new teachers, our early completers performed notably lower than the district average, despite improvement between cycle 1 and 3. Given that these data are consistent across two cycles and three years, the program will consider actions to better prepare completers to manage student behavior.

Measure 1 - KY Impact Survey

Measure 2: (Initial and Advanced) Satisfaction of Employers Stakeholder Involvement (R 4.2, R 5.3, RA 4.1)

Employer Satisfaction

Employer surveys assess satisfaction for both initial certification completers and advance practice completers across programs. Data collection across cycles reflects inconsistent response rates with more robust rates for initial certification candidates across all three cycles. Advanced programs data are less robust with Cycle 2 (2022-2023) advanced programs data failing to meet threshold. Overall, employers report satisfaction with Bellarmine graduates based on previous/current experience and recommend Bellarmine graduates to other school administrators.

At the initial certification level, all three data cycles indicate overall employer satisfaction with specific survey items aligned to InTASC standards. The same survey questions are used across each cycle with the first cycle offering a comparative picture both statewide and with other private institutions across Kentucky. Cycle 1 (2021-2022) data are the result of a statewide survey piloted through the collaboration of Kentucky Association of Colleges for Teacher Education (KACTE) and the Evaluation Center at the University of Kentucky. Cycles 2 (2022-2023) and 3 (2023-2024) provide stand-alone snapshots. The same questions were used, and each was aligned with the Kentucky Teacher Performance Standards/InTASC Standards. This alignment allows for interpretation across cycles.

At the advanced level, data indicate overall employer satisfaction with specific survey items tied to specific professional association standards as relevant to individual fields in cycles 1 (2021-2022) and 3 (2023-2024). Data are discussed in the aggregate before being discussed by relevant standard. We will continue to monitor overall employer satisfaction as well as employer satisfaction in relation to specific program standards (e.g. InTASC Standards, PSEL Standards, ILA Standards). Furthermore, we will continue to recruit higher response rates for advanced program employers. The majority of these employers respond to surveys for our initial certification program completers. The addition of the survey for advanced program completers may feel redundant.

Initial Certification

Employers report positive satisfaction levels of Bellarmine completers across all three cycles of data and across all survey domains.

Cycle 1 (2021-2022) data indicate that employers believe early completers demonstrate proficiency with the InTASC standards through their instructional practices. Completers scored 100% satisfaction in demonstrating the central concepts of their discipline, planning instruction to meet all learner needs, and seeking opportunities to collaborate to support student learning. All other domains scored 75% satisfaction, with Bellarmine graduates outperforming completers from other private institutions in Kentucky in ten out of twelve domains. Cycle 2 data (2022-2023) are consistent with previous cycles with the majority of employers scoring completers as accomplished or exemplary in every domain. Cycle 3 data (2023-2024) maintain this trend with the majority of employers indicating completer performance as accomplished or exemplary on every indicator. 100% of employers rating completers accomplished or exemplary in fostering inclusive learning environments, and more than 90% of completers accomplished or exemplary in creating environments that support individual and collaborative learning and demonstrating the central concepts of the discipline taught. Cycle three data indicate dips in using multiple modalities for instruction along with multiple methods of assessment. These data will be monitored in ongoing cycles. Qualitative comments indicated a desire for students to improve classroom management, support for multilingual learners, and assessment and standards-based instruction. Overall, employers find early completers to be effective classroom teachers. We will continue to monitor the data with a focus on these areas for growth.

Advanced Certification

Employer satisfaction surveys for advanced program graduates ask employers to report candidate preparation in relation to specific program association standards. The Teacher Leader program is aligned to the Teacher Leader Model Standards. The Principal Preparation program is aligned to the Professional Standards of Educational Leaders (PSEL). The Literacy Leader program is aligned with the standards set forth by the International Literacy Association (ILA). Because our completers reflect a small sample size, response rates per program are small. These data will continue to be monitored across cycles. As noted previously, Cycle 2 (2022-2023) survey responses failed to meet the threshold. Discussion reflects Cycle 1 (2021-2022) and Cycle 3 (2023-2024) survey responses.

Teacher Leader Model Standards

Across the two available cycles of data (2021-2022 & 2023-2024), employers indicated high rates of satisfaction for teacher leader completers across all Teacher Leader Model Standards. One hundred percent of employers agree or somewhat agree completers create collaborative environments, use systemic inquiry, promote continuous improvement, model behaviors that promote instructional improvement, optimize school improvement strategies, facilitate collaboration among stakeholders, and advocate for educational policy. We will continue to monitor these data and support increased response rates to document the effectiveness of completers at advanced levels.

Professional Standards of Educational Leaders (PSEL)

Across both cycles of data (2021-2022 & 2023-2024), employers indicated high rates of satisfaction for Ed.S. completers across all PSEL standards with employers agreeing or somewhat agreeing that completers demonstrate these standards in observable ways. One hundred percent of employers agree or somewhat agree completers enact core values for academic success, demonstrate professional standards, support culturally responsive practices, support rigorous instruction, cultivate inclusivity, fosters professional community, positively engages with stakeholders, manage resources well, and practices continuous improvement.

International Literacy Association (ILA) Standards

Across both cycles of data (2021-2022 & 2023-2024), employers indicated high rates of satisfaction for Literacy Leader completers across all ILA standards with employers agreeing or somewhat agreeing that completers demonstrate these standards in observable ways. All respondents agree or somewhat agree that completers demonstrate evidence-based foundations of literacy, employ evidence-based instruction, create inclusive learning environments, utilize appropriate assessment methods, employ multiple modes of materials, and facilitate ongoing professional learning. There were no identified deficiencies across these standards. Qualitative comments indicated a need to focus more on the science of reading and a specific request to prepare new teachers with this in mind. The School of Education has revised literacy methods courses at the initial and advanced levels to better accomplish this request. We will continue to monitor feedback to better understand the impact of these revisions.

Stakeholder Involvement

In order to co-construct a mutually beneficial partnership between the AFTSE and our partner districts, the AFTSE meets as an entire unit twice a year with our Advisory Council on Education (ACE) committee, which includes multiple stakeholders from our partner districts. ACE committee meetings are devoted to a specific need each year identified by both our partner districts and our EPP. A smaller committee, the Bellarmine Educators Stakeholders Team (BEST), which consists of some AFTSE faculty members, administrators, teachers, and recruiters from JCPS and the Archdiocese of Louisville, meets three times a semester to provide input about individual programs, key assessments, and innovations. The AFTSE and our clinical partners share responsibility for the co-construction of key instruments used in the field, big picture curricular decisions, and programmatic changes. Over time, this collaborative process leads to purposeful changes and creates a cohesive experience for our candidates. Summaries of annual stakeholder activity are outlined in the tables below.

Measures of Program Outcomes

Measure 3 (Initial and Advanced): Candidate Competency at Completion (R.3.3)

To measure candidate competency at the initial and advanced levels, we examine licensure and end of program assessments.

Initial Certification

Bellarmine works with ETS Praxis data to track candidate success rates on licensure assessments. Across all three cycles of data at the initial certification level, Bellarmine’s pass rates remain high when the data samples reach reporting threshold. For cycles 1 (2021-2022) and 2 (2022-2023) 100% of our completers passed the required assessments in both traditional and alternative pathways. Cycle 3 (2023-2024) data break this trend, with only 92% of traditional pathway completers passing the required assessments. This is the only instance when Bellarmine data fall below the statewide pass rate. We will continue to monitor this in future cycles. Cycle 3 data were not reported for completers of the alternative pathway as these numbers fell below the reporting threshold. Data are redacted when samples are lower than 10.

Advanced Certification

Across all three cycles of data at the advanced certification level, Bellarmine’s pass rates indicate candidate competency at completion. The Literacy Specialist assessment reflects an 100% pass rate in cycle 2 (2022-2023) and cycle 3 (2023-2024) with no completers in cycle 1. The Principal Preparation assessments reflect 100% pass rate across all three cycles. Bellarmine’s pass rates remain at or above state averages on these assessments.

Beginning in the 2022-2023 academic year, Kentucky adjusted all ETS pass rates to include scores within one standard error of measure (-1 SEM); the passing score on each table reflects this statewide adoption.

Measure 4 (Initial and Advanced): Ability of Completers to be Hired

Bellarmine’s Career Development Center conducts an annual First Destination Survey to collect hiring data. The last three cycles of our annual First Destination Survey indicate Bellarmine students are sought after. The Career Development Center reports strong knowledge rates: 79% in Cycle 3 (2023-2024), 89% in Cycle 2 (2022-2023), 81% response rate in Cycle 1 (2021-2022). Findings are reported as overall, undergraduate, and graduate, which does not clearly differentiate between initial certification and advanced program completers. Still, the overall data indicate that the vast majority of completers are employed with the remainder pursuing continuing education. This trend persists across different demographic profiles.

Initial Certification

Undergraduate data reflect initial certification program completers and are thus a logical indicator of these completers ability to be hired. Cycle 3 (2023-2024) data indicate that 100% of graduates are working, Cycle 2 (2022-2023) data indicate that 87% of education graduates are employed in their field, and Cycle 1 (2021-2022) data indicate that 92% are employed, volunteering, or in the military. For Cycles 1 and 2, the remaining graduates report pursuing continuing education. Jefferson County Public Schools, the local district, is identified as the highest employer of School of Education graduates across each cycle with the Archdiocese and suburban districts ranking second and third.

Advanced Programs

Graduate employment data encompasses advanced certification completers along with MAT graduates. While the representation of each group is not clear, these data establish the ability of our advanced program completers to be hired. Data for Cycle 3 (2023-2024) and Cycle 1 (2021-2022) both indicate that 90% advanced program completers are employed with 10% continuing their education. Cycle 2 (2022-2023) did not specify graduate student employment rates. Given the available data across two cycles, it is clear that Bellarmine’s advanced program completers are working. Because advanced practitioners may not immediately obtain a license upon program completion, we are unable to determine how many of these graduates are working in their most current field. We monitor licensure exam pass rates as these suggest the number of completers who intend to seek employment based on their new credentials. Across all three cycles, these pass rates indicate that some completers would be seeking employment that reflects their new credential. See measure three for these specific data.