This is an excerpt from our upcoming book, LEARNING TO LOVE DATA
Physician Assistant educational programs are facing unprecedented challenges to implement an assessment system to scale. In an era of unprecedented growth of PA programs in the United States the number of experienced faculty has diminished. The stress and rigor of developing a curriculum and implementing a new program stretches faculty to the breaking point. New PA programs face two additional provisional visits in which the ARC-PA requires evidence of full implementation of Appendix 14. Chapter 2, Accreditation Driven Assessment Practices, provides an overview of the full scope of assessment requirements required by the 5th Edition Standards. According to the 14th faculty report published by the Physician Assistant Education Association (PAEA), the experience of faculty nationwide demonstrates the following: 47.5% of all faculty have less than five years’ experience. In comparison, 42.7% of all program directors have less than 10 years of academic experience. The potential impact of less experienced faculty on both developing and established programs is currently unknown. Nonetheless, this trend bears watching due to the rapid increase of new programs. These challenges constitute a legitimate threat to PA program survival in the future. Without major change, existing programs may not be in business 10 years from now. This book is dedicated to reducing these risks and provided much-needed assessment knowledge among faculty nationwide.
Challenges Programs Face
Besides the pressure to create a useful assessment system, established PA programs also face new challenges to keep pace with external expectations. With the incorporation of the 5th Edition Standards the amount of quantitative and qualitative data continues to grow. The old model of “looking at the data each year” has become obsolete. In addition, programs are required by their sponsoring institution to produce annual reports which focus on acquisition of program goals and outcomes. These challenges dictate that programs must implement an annual calendar of data collection, analysis, and developing action plans based upon the required appendix 14 elements. Yet many faculties lack basic skill sets in quantitative analysis and statistical methods.
Increased Number of PA Programs on Probation
On the ARC-PA accreditation actions website a brief exploratory study reveals several interesting trends. Since 2015, 39 PA programs received an adverse commission action of accreditation-probation status. In 2019 nine programs received probation, with an additional nine programs receiving probation in 2020. Although it is difficult to articulate the root cause of every case, there seems to be an increased number of citations related to ongoing program self-assessment.
Although the commission actions website did not list the specific citations given in 2016 and 2017, a list of required reports provides insights about the likely citations. The table below (Programs on probation: citations received) provides a diagrammatic representation of the number of citations related to specific standards.
The following list includes the standards along with the number of citations recorded for each on the accreditation action website.
C1.01 The program must implement an ongoing program self-assessment process that is designed to document program effectiveness and foster program improvement.
Since 2015, nine programs received citations related to the implementation of an ongoing program self-assessment process.
C1.02 The program must apply the results of ongoing program self-assessment to the curriculum and other dimensions of the program.
Since 2015, 11 programs received citations related to applying the results of ongoing program self-assessment to the curriculum and other dimensions of the program.
C2.01 The program must prepare a self-study report as part of the application for continuing accreditation that accurately and succinctly documents the process, application, and results of ongoing program self-assessment.
Since 2015, 20 programs received citations related to preparing a self-study report that accurately and succinctly documents the process, application, and results of program self-assessment.
Programs on probation: Citations received
Based on this brief analysis, the root cause for citations most often seems to be the ability to document the four stages of assessment described in this eBook. This suggests that programs are struggling with the process of drawing conclusions, developing action plans, and following through with the modifications. In simple terms, “closing the loop” is the most troublesome aspect for many programs, a serious problem that threatens the survival of PA programs throughout the nation.
According to the accreditation action website, three PA programs have withdrawn accreditation in 2020. Two of those three had received two previous terms of probation. The third was withdrawn during a regular continuing accreditation evaluation. Although it is too early to assume this is a disturbing trend, the immediate impact on the specific institutions and students enrolled in those programs is enormous, not to mention the financial impact of lost tuition revenue. Regardless, programs need to improve the data and assessment process, but these statistics underscore the threat that programs face if they do not achieve compliance with the ARC-PA accreditation standards.
For purposes of this eBook we will focus on how programs can improve data collection and assessment to ensure compliance with the aforementioned standards. At a time when the number of new programs continues to grow, this issue must be addressed systemically.