CAEP Accountability Measures

The Bellarmine University Annsley Frazier Thornton School of Education (AFTSOE) prioritizes the use of data as part of its assessment and continuous improvement process. The data provided below represents a snapshot summary of survey reports, state and school district data reports, and teacher candidate performance assessment data. Data are acquired from sources including Jefferson County Public Schools (JCPS); the Kentucky Education Professional Standards Board (EPSB); Educational Testing Service (ETS); Bellarmine Career Development Center and the Bellarmine University Office of Institutional Research and Effectiveness. We have provided links to these data to assist prospective and current students, faculty, accrediting bodies, the public, and researchers.

The following programs were reviewed and approved by CAEP during our Fall 2019 accreditation visit:

  • Bachelor of Arts (B.A.) in Education
  • Master of Arts in Teaching (MAT)
  • Master of Arts in Education (MAEd)
  • Doctor of Education in K-12 District Leadership (Ed.D.)
  • Educational Specialist degree in Instructional Leadership and School Administration (Ed.S.)
  • Rank I individualized program in Curriculum and Instruction

Measures of Program Impact

Measure 1: (Initial) Completer Effectiveness and Impact on P-12 Learning and Development (CAEP Component R4.1)

Measure 1 (Initial): Completer Effectiveness (R4.1)

The greatest number of our completers obtain work in the Jefferson County Public School (JCPS) system. To understand the teaching effectiveness of our 1st-3rd year completers, we examined data from various sources within JCPS. To surmise the impact our 1st-3rd year teachers have on P-12 learning and development, we examine student growth in reading and math. To surmise our 1st-3rd year completers effectiveness in applying professional knowledge, skills, and dispositions, we examine the results of two surveys: the Kentucky Impact Survey at the district and school levels. District assessment measures changed each cycle, making direct comparisons across cycles difficult. We hope the district continues the collection of consistent measures that will allow us to compare data across cycles.

Contribution to K-12 Student Learning Growth

For cycle 1 (2018-2019), the district provided mean MAP scores in reading and math at the elementary level. The sample size did not meet the required threshold for middle and high school scores. Due to the COVID 19 pandemic, district level and state level data were not available for Cycle 2 (2019-2020). Kentucky received an assessment waiver and exempted districts from accountability requirements in the 2019-2020 school year. For cycle 3 (2020-2021), the district provided scale scores on the Kentucky State Assessment in reading and math.

To infer completer impact on K-12 student learning, we compare the data reflecting the performance of students taught by our 1st-3rd year completers with the growth of early career teachers in the district. While the data for cycle 1 and cycle 3 reflect different assessments, both offer an opportunity to examine the impact of our 1st-3rd year completers as compared to early career teachers.

Our completers perform at similar levels to other early career teachers. With cycle 1, math scores approximate the district average for early career teachers, and reading scores are a bit lower in early career Bellarmine teachers’ classrooms. Within cycle 3, elementary reading scores approximate district average while math scores fall a bit below this mean. Middle school reading and math scores approximate the district means and high school math also approximates the district mean. The sample size for high school reading did not meet the required threshold. We continue to monitor these data as trends emerge and hope future data enables us to compare data across cycles.

Completer Effectiveness in Applying Professional Knowledge, Skills, and Dispositions

We examined the Kentucky Impact Survey for two components: Educating all Students and Managing Student Behavior. The data below reflect comparisons between district level data and elementary school data where our early career educators teach. We were not able to procure data for cycle 1 as the state discontinued the Kentucky Teacher Internship Program and the New Teacher Survey, both of which we collected for previous cycles. For cycle 2, we were able to identify one elementary school where four of our teachers work and examined data for early career teachers within this school. This school presents as a single case through which we can consider the effectiveness of our early completers. For cycle 3, we were able to identify two elementary schools with three of our early career teachers.

These data provide partial a snapshot of effectiveness for early completers. At schools with notable numbers of Bellarmine early completers, our completers perform consistently with early career teachers across the district when it comes to educating all students and better than district comparisons when it comes to managing data.

Measure 2: (Initial and Advanced) Satisfaction of Employers Stakeholder Involvement (CAEP Components R4.2, R5.3, and RA4.1)

Employer Satisfaction

Employer survey assesses satisfaction for both initial certification completers and advance practice completers across programs. Across all three cycles of data and across initial and advanced certification levels, 100% of responding employers report they would hire another Bellarmine graduate based on previous/current experience and recommend Bellarmine graduates to other school administrators. At the initial certification level, data indicate overall employer satisfaction with specific survey items aligned to InTASC standards. At the advanced level, data indicate overall employer satisfaction with specific survey items tied to specific professional association standards as relevant to individual fields. Data are discussed in the aggregate before being discussed by relevant standard. We will continue to monitor overall employer satisfaction as well as employer satisfaction in relation to specific program standards (e.g. InTASC Standards, PSEL Standards, ILA Standards)

Initial Certification

Employers report high satisfaction levels of Bellarmine completers across cycles; employers report satisfaction of Bellarmine completers across all survey domains. Specifically, survey results indicate that Bellarmine completers demonstrate subject matter knowledge in their area of licensure and academic knowledge in their field. One hundred percent of employers agree that our completers demonstrate effective self-evaluation in relation to teaching Kentucky’s learning goals; defining clear professional growth plans based on self-assessment; effective collaboration with stakeholders; creation of emotionally and physically safe learning environments; and high sense of moral, social, and ethical responsibilities as these relate to the field. Over 90% of employers report that Bellarmine graduates work well with people of diverse backgrounds, respond to student behavior in respectful and appropriate ways, communicate learning assessment results with stakeholders, utilize technology in effective ways, and understand the role of effective teaching on their school and community. These results remain consistent across cycles.

Data also indicate that employers believe early completers demonstrate proficiency with the InTASC standards through their instructional practices. Highest satisfaction items include 100% of employers agreeing completers implement developmentally appropriate instruction, build inclusive learning environments, and support collaborative learning for students. Just over 85% of employers agree completers implement rigorous instruction, use multiple learning strategies, employ diverse perspectives, implement meaningful learning experiences, promote critical thinking, allow for content mastery, use multiple assessment methods, and engage in meaningful professional development. Seventy-five percent of employers agree completers seek appropriate professional leadership roles. Overall, employers find early completers to be effective classroom teachers.

Advanced Certification

In addition to general satisfaction items, employer satisfaction surveys for advanced program graduates ask employers to report candidate preparation in relation to specific program association standards. The Teacher Leader program is aligned to the Teacher Leader Model Standards. The Principal Preparation program is aligned to the Professional Standards of Educational Leaders (PSEL). The Literacy Leader program is aligned with the standards set forth by the International Literacy Association (ILA). Because our completers reflect a small sample size, response rates per program are small. These data will continue to be monitored across cycles.

Teacher Leader Model Standards

Employers indicated high rates of satisfaction for teacher leader completers across all Teacher Leader Model Standards with one exception. One hundred percent of employers agree completers create collaborative environments, promote continuous improvement, model behaviors that promote instructional improvement, optimize school improvement strategies, and facilitate collaboration among stakeholders. Eighty percent of employers agree that completers utilize systemic inquiry and advocate for educational policy with the other 20% agreeing or somewhat agreeing on this item. This reflects a positive increase from data in cycle 2.

Professional Standards of Educational Leaders (PSEL)

Employers indicated high rates of satisfaction for Ed.S. completers across all PSEL standards with employers agreeing or somewhat agreeing that completers demonstrate these standards in observable ways. Highest ratings reveal completers support rigorous instruction, cultivate effective school communities, demonstrate professional standards with personnel, foster professional communities that support student success, and practice continuous improvement. Eighty percent of employers agree that completers enact core values, demonstrate professional standards, and engage stakeholders effectively. Sixty percent of employers agree that completers support culturally responsive practices and demonstrate the ability to manage resources well with the other 40% somewhat agreeing to each.

International Literacy Association (ILA) Standards

Employers indicated high rates of satisfaction for Literacy Leader completers across all ILA standards with employers agreeing or somewhat agreeing that completers demonstrate these standards in observable ways. All respondents agree or somewhat agree that completers demonstrate evidence-based foundations of literacy, employ evidence-based instruction, create inclusive learning environments, utilize appropriate assessment methods, employ multiple modes of materials, and facilitate ongoing professional learning. There were no identified deficiencies across these standards.

Stakeholder Involvement

In order to co-construct a mutually beneficial partnership between the AFTSE and our partner districts, the AFTSE meets as an entire unit twice a year with our Advisory Council on Education (ACE) committee, which includes multiple stakeholders from our partner districts. ACE committee meetings are devoted to a specific need each year identified by both our partner districts and our EPP. A smaller committee, the Bellarmine Educators Stakeholders Team (BEST), which consists of some AFTSE faculty members, administrators, teachers, and recruiters from JCPS and the Archdiocese of Louisville, meets monthly to provide input about individual programs, key assessments, and innovations. The AFTSE and our clinical partners share responsibility for the co-construction of key instruments used in the field, big picture curricular decisions, and programmatic changes. Over time, this collaborative process leads to purposeful changes and creates a cohesive experience for our candidates.

Measures of Program Outcomes

Measure 3 (Initial and Advanced): Candidate Competency at Completion (R.3.3)

To measure candidate competency at the initial and advanced levels, we examine licensure and end of program assessments.

Initial Certification

Bellarmine works with ETS Praxis data to track candidate success rates on licensure assessments. Across all three cycles of data at the initial certification level, Bellarmine’s pass rates range from 97% to 100% for cycles 1-3. Data for our alternative certification students are incomplete as samples lower than ten are redacted. For cycles 1 and 3, we had fewer than ten alternative certification completers. For cycle 2, 100% of our alternative certification students passing required licensure assessments.

Advanced Certification

Across all three cycles of data at the advanced certification level, Bellarmine’s pass rates indicate candidate competency at completion. The Literacy Specialist assessment rates range from 88% to 100%. The Principal Preparation assessments range from 75% to 100% for the school Leaders assessment and hold at 100% for the specialty test of instructional administrative practices. Bellarmine’s pass rates remain at or above state averages on these assessments.

Measure 4 (Initial and Advanced): Ability of Completers to be Hired

Initial Certification

Bellarmine’s Career Development Center conducts an annual First Destination Survey to collect hiring data. For Cycle 3 data, the Career Development Center reports 87% response rate. These data indicate that 67% of our initial certification completers are employed in their field 33% are pursuing graduate education. Jefferson County Public Schools and Bullitt County Public Schools are the two biggest employers of our initial certification graduates.

Advanced Certification

Data for cycle 3 advanced program completers reveal 95% employment with completers holding positions as a classroom teacher, administrator, and instructional coach. Nine percent of these completers are also pursuing a second graduate degree. Because advanced practitioners may not immediately obtain a license upon program completion, we monitor licensure exam pass rates. These pass rates suggest the number of completers who intend to seek employment based on their new credentials.

Employer satisfaction survey indicates that 100% of responding employers would hire another Bellarmine graduate based on previous/current experience and recommend Bellarmine graduates to other school administrators. These data remain consistent across all three cycles of data and across initial and advanced programs.