5th Edition Standards
On September 1, 2020, all Physician Assistant programs officially transitioned to the ARC-PA’s 5th Edition Standards. Since this process cannot happen overnight, PA programs are now deciding which quality assurance process should be implemented in order to organically change their program processes and procedures to capture the essence of the 5th Edition Standards.
Changes and Updates
The 5th Edition Standards include several major improvements from the 4th Edition Standards. For instance, clarity of the standards has been enhanced. The annotations from the 4th Edition Standards, known to change over time unbeknownst to PA programs, have also been removed. Notably, many of the standards have changed language when comparing the same letters and numbers, which can easily be cross walked to begin incorporating.
The most impactful change made within the 5th Edition Standards is the self-assessment process, defined by Standard C1.01. Since the changes are quite nuanced, we’ll examine them by defining the basic standard and sub-standards from C1.01, then combine the self-study report language and provide a few “pearls of wisdom” about how your PA program can begin working toward compliance.
Standard C1.01
Within the ARC-PA’s 5th Edition Standards, Standard C1.01 states:
C1.01 The program must define its ongoing self-assessment process that is designed to document program effectiveness and foster program improvement. At a minimum, the process must address:
a) administrative aspects of the program and institutional resources,
b) effectiveness of the didactic curriculum,
c) effectiveness of the clinical curriculum,
d) preparation of graduates to achieve program-defined competencies,
e) PANCE performance,
f) sufficiency and effectiveness of principal and instructional faculty and staff, and
g) success in meeting the program’s goals.
Now, let’s unpack each sub-standard combined with the self-study report language:
a) Administrative aspects of the program and institutional resources
Appendix 14B
Institutional resources in support of the program
Financial resources
Human resources
Physical resources
Technology resources and support
Safety and security, and student services (defined in the glossary as: academic advising, tutoring, career services, financial aid, student health, computing and library resources and access)
Support for faculty development, clinical site development, curriculum design and course selection, program assessment, and program diversity and inclusion
Program administrative aspects
Policies/procedures
Admissions processes and outcomes
Pearls of Wisdom
When constructing your assessment instruments, be sure to include all of the elements listed above. Consider using an expanded faculty and staff survey to evaluate each of the elements; this can be captured through quantitative and qualitative analysis. Or, consider capturing the same data from students’ perspectives.
b) Effectiveness of the didactic curriculum
Appendix 14C
Student evaluation of didactic courses and instructors
The number of final didactic course grades at C or below
Student attrition* and remediation in didactic courses
Pearls of Wisdom
Essentially, this combines the previous Appendices 13B, 13C, 13D, and 13E. Remember to follow an equally robust process for each of these elements. The new templates now provide an avenue to include modifications, strengths, and areas needing improvement within each document.
c) Effectiveness of the clinical curriculum
Appendix 14D
Student evaluation of clinical rotations and preceptors
The number of final clinical rotations grade of C or below
Student attrition* and remediation in clinical rotations
Pearls of Wisdom
The de-emphasis of preceptor evaluation of students is a major pivot in this section. Effectively, the preceptor and SCPE evaluations from Appendices 13B and 13C are included within this section. Be sure to re-examine how your PA program is tracking student remediation practices within the clinical year.
d) Preparation of graduates to achieve program-defined competencies
Appendix 14E
Summative evaluation performance
Exiting student/graduate feedback
Faculty evaluation of the curriculum to assess its ability to prepare students to achieve program defined competencies
Pearls of Wisdom
There are a few major changes here, including an increased emphasis on presenting summative evaluation in much greater detail. PA programs are required to correlate these elements with the curriculum and the competencies. An emphasis is also now put on evaluating program-defined competencies, meaning PA programs need to incorporate graduation competencies within the assessment process.
e) PANCE performance
Appendix 14F
Data analysis related to PANCE outcomes is to include (if not already analyzed in another Appendix) but not limited to correlation of PANCE outcomes and:
Admissions criteria as a predictor of success
Course outcomes
Course and instructor evaluations by students
Program instructional objectives, learning outcomes, and breadth and depth of the curriculum
Student summative evaluation results
Remediation practices and results
Student progress criteria and attrition data
Feedback from students who were unsuccessful on PANCE, if available
Preceptor and graduate feedback (employer feedback is not required but may be helpful if available to the program)
Pearls of Wisdom
This section has reinforced the sheer depth and breadth of assessment required related to PANCE performance. Regardless of 100% pass rate, all of the aforementioned items are expected and required in Appendix 14F. This analysis can be done descriptively, but some of it can be accomplished through appropriate statistical analysis through regression and correlation. We examined this in more detail in our post How to Help Students at Risk: Preparing for PANCE.
f) Sufficiency and effectiveness of principal and instructional faculty and staff
Appendix 14G
Provide a narrative describing the factors used to determine effectiveness of principal and instructional faculty in meeting the academic needs of enrolled students and managing the administrative responsibilities of the program. Describe how the program collects data related to those factors to determine effectiveness of program faculty in meeting the program’s needs.
Pearls of Wisdom
According to Appendix 13I, PA programs are to evaluate both sufficiency and effectiveness. This can be accomplished through faculty evaluations by students and through self-assessment by faculty. To evaluate the program’s administrative process, gather both faculty and student perspectives.
g) Success in meeting the program’s goals
Appendix 14H
Provide a tabular or graphic display of data collected by the program for each of its goals for the past three academic years. If already provided in another appendix, reference that data in the analysis narrative
Quantitative data must be reported in aggregate and displayed in tables or graphs that directly support the analysis. (Do not provide raw data)
Qualitative data themes used in the analysis must be reported and summarized in the narrative or displayed in an appended document. (Do not provide raw data)
Data should be presented in a way that allows comparison across years and appreciation of trends over time
Reference other appendices of the SSR as needed
Pearls of Wisdom
This section certainly emphasizes the level of granularity required in evaluating achievement of program goals, which begs the questions: How measurable are the PA program’s current goals? Do they need to be modified? Another key feature is the longitudinal evaluation of each of these. If your PA program is not achieving the goals based on the benchmark, they must be modified promptly. This also suggests a more robust display of data on the PA program’s website.
Conclusion
It is essential PA programs incorporate each of the elements of assessment during the first year of transitioning from the 4th Edition Standards to the 5th Edition Standards. It is vital to look at the current system of data collection regarding its viability and robustness. At the very least, there should be a well-defined process for tabulating and analyzing the data on an ongoing basis. This must result in faculty reviewing and drawing conclusions multiple times per year; the old model of the annual retreat reviewing data comprehensively has become obsolete.
More positively, when fully implemented, this process will be a highly effective quality assurance process, able to drive change within the PA program in real time. Remember, these changes must be captured in program minutes on an ongoing basis. When it’s time to complete the new 5th Edition Standards’ self-study report, this should be a celebration of the program’s commitment to excellence.