CAEP Accountability Measures

The Bellarmine University Annsley Frazier Thornton School of Education (AFTSOE) prioritizes the use of data as part of its assessment and continuous improvement process. The data provided below represents a snapshot summary of survey reports, state and school district data reports, and teacher candidate performance assessment data. Data are acquired from sources including Jefferson County Public Schools (JCPS); the Kentucky Education Professional Standards Board (EPSB); Educational Testing Service (ETS); Bellarmine Career Development Center and the Bellarmine University Office of Institutional Research and Effectiveness. We have provided links to these data to assist prospective and current students, faculty, accrediting bodies, the public, and researchers.

The following programs were reviewed and approved by CAEP during our Fall 2019 accreditation visit:

  • Bachelor of Arts (B.A.) in Education
  • Master of Arts in Teaching (MAT)
  • Master of Arts in Education (MAEd)
  • Doctor of Education in K-12 District Leadership (Ed.D.)
  • Educational Specialist degree in Instructional Leadership and School Administration (Ed.S.)
  • Rank I individualized program in Curriculum and Instruction

Measures of Program Impact

Measure 1: (Initial) Completer Effectiveness and Impact on P-12 Learning and Development (CAEP Component R4.1)

Measure 1 (Initial): Completer Effectiveness (R4.1) 

The greatest number of our completers obtain work in the Jefferson County Public School (JCPS) system. To understand the teaching effectiveness of our 1st-3rd year completers, we examine data from various sources within JCPS.  To surmise the impact our 1st-3rd year teachers have on P-12 learning and development, we examine student growth in reading and math. To surmise our 1st-3rd year completers effectiveness in applying professional knowledge, skills, and dispositions, we examine the results of a statewide survey: the Kentucky Impact Survey at district and school levels.  District and state data measures changed across three cycles, making direct comparisons across cycles difficult. We hope the district and state continue the collection of consistent measures that will allow us to compare data across cycles. 

Contribution to K-12 Student Learning Growth 

Due to the COVID 19 pandemic, district level and state level data were not available for cycle 1 (2019-2020). Kentucky received an assessment waiver and exempted districts from accountability requirements in the 2019-2020 school year. For cycle 2 (2020-2021) and cycle 3 (2021-2022), the district provided scale scores on the Kentucky State Assessment in reading and math. To infer completer impact on K-12 student learning, we compare the data reflecting the performance of students taught by our 1st-3rd year completers with the growth of early career teachers in the district. This comparison offers an opportunity to examine the impact of our 1st-3rd year completers as compared to early career teachers across the district. 

Our completers perform at similar levels to other early career teachers with reading scores trending better than math scores. Within cycle 2, elementary reading scores approximate district average while math scores fall a bit below this mean. Middle school reading and math scores approximate the district means and high school math also approximates the district mean. The sample size for high school reading did not meet the required threshold. Within cycle 3, reading scores at elementary and high school indicate Bellarmine early completers are more successful than other early career teachers throughout the district. Middle school reading scores fall just below the district mean. with fewer than four years’ experience. Overall district means exceed Bellarmine completers by one to two points in math, indicating an area for continued improvement. We continue to monitor these data with the goal of comparing data across cycles.

Completer Effectiveness in Applying Professional Knowledge, Skills, and Dispositions 

We examined the Kentucky Impact Survey for two components: Educating all Students and Managing Student Behavior. This survey is administered every two years, reflecting completer effectiveness for cycle 1 (2019-2020) and cycle 3 (2020-2021). There were no data available for cycle 2 (2020-2021). Cycle 1 data below reflect comparisons between district level data and elementary school data where our early career educators teach. For cycle 3, we were able to identify two schools where three or more of our teachers work and examined data for early career teachers within these schools. These schools present two cases through which we can consider the effectiveness of our early completers. 

These data provide a partial snapshot of effectiveness for early completers. At schools with notable numbers of Bellarmine early completers, our completers perform consistently with early career teachers across the district when it comes to educating all students. The difference in cycle 1 and cycle 3 data for managing student behavior calls for continued monitoring. Managing student behavior is an essential component of teaching and while these data are not robust enough to warrant programmatic change, we will attend to this domain as additional data are collected. 

Measure 2: (Initial and Advanced) Satisfaction of Employers Stakeholder Involvement (CAEP Components R4.2, R5.3, and RA4.1)

Employer Satisfaction

Employer surveys assess satisfaction for both initial certification completers and advance program completers across programs. Overall, employers report satisfaction with Bellarmine graduates based on previous/current experience and recommend Bellarmine graduates to other school administrators. At the initial certification level, cycle 1 (2019-2020) & 2 (2020-2021) data indicate overall employer satisfaction with specific survey items aligned to InTASC standards. Cycle 3 (2021-2022) data are not yet available. In spring of 2023, The Annsley Frazier Thornton School of Education collaborated with the Kentucky Association of Colleges for Teacher Education (KACTE) and the Evaluation Center at the University of Kentucky to pilot a new statewide survey of employer satisfaction for initial certification program completers. These data will be available later this summer and updated by the end of August 2023.   

At the advanced level, data indicate overall employer satisfaction with specific survey items tied to specific professional association standards as relevant to individual fields. Data are discussed in the aggregate before being discussed by relevant standard. We will continue to monitor overall employer satisfaction as well as employer satisfaction in relation to specific program standards (e.g. InTasc Standards, PSEL Standards, ILA Standards)

Initial Certification

Employers report positive satisfaction levels of Bellarmine completers across two cycles of data and across all survey domains.

Specifically, cycle 1 (2019-2020) and cycle 2 (2020-2021) survey results indicate that Bellarmine completers demonstrate subject matter knowledge in their area of licensure and academic knowledge in their field. One hundred percent of employers agree that our completers demonstrate effective self-evaluation in relation to teaching Kentucky’s learning goals; defining clear professional growth plans based on self-assessment; effective collaboration with stakeholders; creation of emotionally and physically safe learning environments; and high sense of moral, social, and ethical responsibilities as these relate to the field. Over 90% of employers report that Bellarmine graduates work well with people of diverse backgrounds, respond to student behavior in respectful and appropriate ways, communicate learning assessment results with stakeholders, utilize technology in effective ways, and understand the role of effective teaching on their school and community.

Cycle 1 (2019-2020) and 2 (2020-2021) data also indicate that employers believe early completers demonstrate proficiency with the InTASC standards through their instructional practices. Highest satisfaction items include 100% of employers agreeing completers implement developmentally appropriate instruction, build inclusive learning environments, and support collaborative learning for students. Just over 85% of employers agree completers implement rigorous instruction, use multiple learning strategies, employ diverse perspectives, implement meaningful learning experiences, promote critical thinking, allow for content mastery, use multiple assessment methods, and engage in meaningful professional development. Seventy-five percent of employers agree completers seek appropriate professional leadership roles. Overall, employers find early completers to be effective classroom teachers. Cycle 3 (2021-2022) data will be available by the end of August 2023. 

Advanced Certification

In addition to general satisfaction items (100% of employers would hire another Bellarmine graduate and recommend a Bellarmine graduate to another school leader), employer satisfaction surveys for advanced program graduates ask employers to report candidate preparation in relation to specific program association standards. The Teacher Leader program is aligned to the Teacher Leader Model Standards. The Principal Preparation program is aligned to the Professional Standards of Educational Leaders (PSEL). The Literacy Leader program is aligned with the standards set forth by the International Literacy Association (ILA).  Because our completers reflect a small sample size, response rates per program are small. These data will continue to be monitored across cycles.

Teacher Leader Model Standards

Across all three cycles of data, employers indicated high rates of satisfaction for teacher leader completers across all Teacher Leader Model Standards. One hundred percent of employers agree or somewhat agree completers create collaborative environments, use systemic inquiry, promote continuous improvement, model behaviors that promote instructional improvement, optimize school improvement strategies, and facilitate collaboration among stakeholders. Across cycle 2 (2020-2021) and 3 (2021-2022), 100% of employers agree that or somewhat agree completers advocate for educational policy. This reflects a positive increase from data in cycle 1 (2019-2020).

Professional Standards of Educational Leaders (PSEL)

Across all three cycles of data, employers indicated high rates of satisfaction for Ed.S. completers across all PSEL standards with employers agreeing or somewhat agreeing that completers demonstrate these standards in observable ways. One hundred percent of employers agree or somewhat agree completers enact core values for academic success, demonstrate professional standards, support culturally responsive practices, support rigorous instruction, cultivate inclusivity, fosters professional community, positively engages with stakeholders, manage resources well, and practices continuous improvement.  

International Literacy Association (ILA) Standards

Across all three cycles of data, employers indicated high rates of satisfaction for Literacy Leader completers across all ILA standards with employers agreeing or somewhat agreeing that completers demonstrate these standards in observable ways. All respondents agree or somewhat agree that completers demonstrate evidence-based foundations of literacy, employ evidence-based instruction, create inclusive learning environments, utilize appropriate assessment methods, employ multiple modes of materials, and facilitate ongoing professional learning. There were no identified deficiencies across these standards.

Stakeholder Involvement

In order to co-construct a mutually beneficial partnership between the AFTSE and our partner districts, the AFTSE meets as an entire unit twice a year with our Advisory Council on Education (ACE) committee, which includes multiple stakeholders from our partner districts. ACE committee meetings are devoted to a specific need each year identified by both our partner districts and our EPP. A smaller committee, the Bellarmine Educators Stakeholders Team (BEST), which consists of some AFTSE faculty members, administrators, teachers, and recruiters from JCPS and the Archdiocese of Louisville, meets three times a semester to provide input about individual programs, key assessments, and innovations. The AFTSE and our clinical partners share responsibility for the co-construction of key instruments used in the field, big picture curricular decisions, and programmatic changes. Over time, this collaborative process leads to purposeful changes and creates a cohesive experience for our candidates. Summaries of annual stakeholder activity is outlined in the tables below.

Measures of Program Outcomes

Measure 3 (Initial and Advanced): Candidate Competency at Completion (R.3.3)

To measure candidate competency at the initial and advanced levels, we examine licensure and end of program assessments.

Initial Certification

Bellarmine works with ETS Praxis data to track candidate success rates on licensure assessments. Across all three cycles of data at the initial certification level, Bellarmine’s pass rates is 100% for cycles 1-3 when the data samples reach reporting threshold. Data for our alternative certification students are incomplete as cycle 2 samples are lower than ten and thus redacted. For cycles 1 and 3, 100% of our alternative certification students passed the required licensure assessments. For cycles 1, 2, and 3, our traditional certification students passed the required licensure assessments. Data across pathways and cycles exceed statewide pass rates.

Advanced Certification

Across all three cycles of data at the advanced certification level, Bellarmine’s pass rates indicate candidate competency at completion. The Literacy Specialist assessment reflects a 100% pass rate. The Principal Preparation assessments range from 75% to 100% with the most current cycle reflecting a 100% pass rate. Bellarmine’s pass rates remain at or above state averages on these assessments.

Measure 4 (Initial and Advanced): Ability of Completers to be Hired

Bellarmine’s Career Development Center conducts an annual First Destination Survey to collect hiring data. For Cycle 3 data, the Career Development Center reports 86% response rate for undergraduate completers and an 81% response rate for graduate completers. Overall, data indicate the vast majority of completers are employed with the remainder pursuing continuing education. This trend persists across different demographic profiles.

Initial Certification

Cycle 3 (2021-2022) data indicate that 92% of our undergraduate completers are employed, volunteering, or in the military; 8% are pursuing graduate education. Of the 92% employed, 99% of these are working as teachers. Jefferson County Public Schools is identified as the highest employer of School of Education graduates with the Archdiocese and Oldham County Schools also employing multiple graduates.

 

Advanced Certification

Data for cycle 3 (2021-2022) advanced program completers reveal 90% employment and 10% pursuing a second graduate degree. Because advanced practitioners may not immediately obtain a license upon program completion, we monitor licensure exam pass rates. These pass rates suggest the number of completers who intend to seek employment based on their new credentials.  

Employer satisfaction survey indicates that 100% of responding employers would hire another Bellarmine graduate based on previous/current experience and recommend Bellarmine graduates to other school administrators. These data remain consistent across all three cycles of data and across initial and advanced programs.