Self-Study Report, Appendix 13 F
The new Self-Study Report (SSR) Appendix 13 F (also from the 5th Edition standards) lists several guidelines spotlighting necessary data analysis:
“Data analysis related to PANCE outcomes is to include, but not limited to, correlation of PANCE outcomes and the following:
Admissions criteria as a predictor of success
Course and instructor evaluations by students
Program instructional objectives, learning outcomes, and breadth and depth of the curriculum
Student summative evaluation results
Student progress criteria and attrition data
Feedback from students who are unsuccessful passing PANCE
Preceptor and graduate feedback”
Understanding these standards is vital to carrying them out. In order to gain increased clarity of the requirements, we’re breaking down each of these items and looking at how to comply with them. In Part 1, we reviewed Admissions Criteria as a Predictor of Success and Course Outcomes. In Part 2, we reviewed the Student Summative Evaluation Results. In Part 3, we will look at Student Progress Criteria and Attrition Data.
Student Progress Criteria
Although adherence to the standards is a requirement, each PA program has the right to choose how they meet these elements. Regarding Student Progress Criteria, programs must define what level of performance is required to graduate from the PA program.
Each specific requirement for progression and graduation can be compared to overall PANCE results. For example, consider how programmatic GPA correlates with PANCE scores. It would seem logical to assume this would be a positive linear relationship, but it depends on the program’s grading criteria; certain programs use pass/fail requirements. Other courses may not be good matches in terms of learning domains, thereby reducing potential correlation. Thus, it is important to choose which one of the academic courses could be considered a predictor for student success.
Annually, I have analyzed all academic elements, starting with the prerequisite requirements all the way through graduation. By conducting a multivariable analysis, PA programs can determine which one of these variables has a strong relationship with PANCE. To take it another step further, when considering student progress criteria and attrition data, these elements are irrevocably connected.
When considering attrition data, PA programs can examine the progression requirements throughout the program to look for problem areas. Are there specific courses that seem to correlate strongly with PANCE results? Do students most often fail these classes? Certainly, this is a diagnostic approach, taken to preemptively identify students whose academic performances are in the bottom quartile in the aforementioned classes.
PA Programs can also analyze data retrospectively when considering attrition data, going back several years. This may provide valuable insights that can better define the program’s academic intervention programs. This analysis should drive academic practices that lower future attrition rates. Thus, if students get lower grades in the strong predictor courses, one method might involve developing an academic improvement plan to provide them with additional mentoring and skill development.
Determining At-Risk Students
Although parametric statistics can be helpful, it is often still very insightful to analyze every student who has not yet passed the PANCE. This means comparing performance in all academic elements with classmates who did pass the PANCE on their first attempt. Because the number of students who fail on the first attempt is often very low, I consider students who score 400 or below to be low PANCE performers (LPPs). With this in mind, a PA Program can take this cross-section of students and compare performance involving all academic elements and progression requirements to students who were higher achievers. This helps PA Programs generate a profile of prospective risk, as well as provide insights about early identification of future at-risk students.
In my experience, looking at the interrelationship of all program requirements and the students’ PANCE scores provides insights that can guide and modify remediation practices. Analyzing student performance prior to PA School also plays a powerful role.
I use a method that involves taking five years of retrospective data, then analyzing all the prerequisite requirements. The independent variable is the PANCE score. This has been problematic with the exception of students who have GPAs in the top tier of all previous students. Looking at other elements, such as healthcare experience or comprehensive admission scores, has not yielded a clear relationship with future performance. This brings up an important concept for PA Programs that have a mission-driven focus regarding admissions. For students at the lower tier of undergraduate GPA, it may be advisable to consider a pre-matriculation program or develop a focused remediation at the very start of the program.
I’ve also taken a preventive approach when considering lower attrition rates. Over the past twenty-eight years as a PA Educator, I’ve seen students experience an increased struggle to enter graduate PA Programs. The PA curriculum has increased academic requirements that are more accelerated, more in-depth, and more in breadth. Many undergraduate students lack basic skills, such as test-taking skills, time management, focused study skills, and even reading skills. For the past fifteen years, I have worked to provide a focused PA Program study skill seminar series during the first few weeks of the program. Read more about it here.
In summary, programs can take both retrospective and prospective approaches when analyzing the relationship between PANCE and Student Progress and Attrition. This process can be reframed as a program research project, which can be rich with potential data for presentation and publication. Moreover, reducing attrition and ensuring the future success of our students is first in all of our hearts. I hope that these suggestions will be helpful to you as you consider the next self-study report.