Last week, we discussed several continuous action items an inaugural PA program director must tackle when developing a new PA program. We covered curriculum development, the ARC-PA application completion, and facility development. Now, we’ll be focusing on the development of the program’s assessment system.
Developing an assessment system for your PA program
While all the continuous action items we’re discussing are very important, the most crucial one is creating a highly functional assessment process. Designing the assessment process now will make the rest of the process run smoother later on.
During the initial provisional application, the ARC-PA does not require data collection and analysis; however, PA programs still must demonstrate all their pieces are in place. Consider the assessment a form of action research; don’t focus too much on satisfying the ARC-PA at this point. If the assessment is incorporated correctly, your PA program will have an advanced system of continuous quality improvement. (There are also opportunities for scholarship hiding within the forthcoming spreadsheets!)
Before students even arrive, there must a paradigm of data-driven decision-making. According to the 4th Edition Standards, PA programs are required to provide the following elements in Appendix 13 of the Self-Study report (SSR):
Appendix K Timeline
Appendix 13 I Sufficiency of staff and faculty
Program-generated schematic demonstrating faculty and staff sufficiency
Appendix 13 SSR, which requires a detailed overview of your assessment process
Let’s review each of these items in detail, including a look at illustrative examples of actual instruments used successfully during this process.
Appendix K Timeline
The following is an excerpt taken from the ARC-PA:
Complete the table in the Appendix 13 K Timeline document below representing the program’s self-assessment process. The table must include the timing of data collection and analysis. For example: “end of each semester,” “annually in August,” etc. The timing of data collection and analysis may be listed separately if appropriate. For example: “Data collected March—April. Analysis annually in May.” Indicate who (job title, committee name, etc.) is responsible for study of the data.
Unfortunately, many PA programs underestimate the requirements for successfully implementing a highly functional assessment. In order to comprehend what’s ahead, Massey & Martin, LLC recommends this effective approach to conceptualize the process: First, lay out all aspects of Appendix 13 for the provisional monitoring visit. Next, determine the disposition of data collection and analysis. Then, decide which committee will be involved in developing action items and implementing appropriate modifications. Remember, assessment practices must be accounted for within the faculty workload and future hires.
Appendix 13 I Sufficiency of Staff and Faculty
It is your responsibility to determine the sufficiency of your faculty and staff once the program is implemented. Remember, you must look at least three years into the future. If you have a 28-month program during the third year of operations, you’ll have three overlapping cohorts. This requires prospective analysis of faculty workloads, student faculty FTEs, and sufficiency of staff members to effectively educate the PA students. Using benchmark data from PAEA can be helpful to provide a comparison for similar size programs regarding enrollment. You are also responsible for developing a schematic to longitudinally demonstrate this data.
Example of an appropriate program-generated schematic chart demonstrating faculty and staff sufficiency
Appendix 13 SSR, which requires a detail overview of your assessment process
Conceptualizing the assessment process in your program requires a thorough understanding of how each component will perfectly coordinate with one another. Achieving this is often a major difficulty for PA programs because it also involves a comprehensive understanding of the database that is to be gathered once the program is operationalized. Then the process must facilitate and achieve four elements of analysis as described by the ARC-PA:
Regular, ongoing collection of data: For ease of use and interpretation, the collected quantitative and qualitative data must be clearly displayed in tables and charts
Analysis of data: This includes discussing and interpreting the correlations and trends relating the data to the expectations or issues of the program; this is to be demonstrated by succinctly written narratives highlighting the correlations, relationships, and trends
Application of results and the development of conclusions based on study of the data: These must be succinctly stated, showing the link between analysis and conclusions; this includes identification of strengths, as well as areas in need of improvement
Development of an action plan to operationalize the conclusions: Action plans, too, must be succinctly stated and should logically result from the conclusions drawn from critical analysis of data
New and developing PA programs often realize they are lacking adequate resources and time to ensure all four components are completed; it’s certainly a monumental task to teach the curriculum for the first time. Thus, Massey & Martin, LLC advocates for a highly efficient and functional committee structure able to facilitate decision-making and record the appropriate modifications.
Massey & Martin, LLC provide an example of an appropriate narrative within the SSR here, personally drafted by Dr. Scott Massey. Also included are diagrams to enhance understanding of the hypothetical program’s assessment system.
As the inaugural PA program director, developing the SSR is considered of utmost importance on your task list. Creating and maintaining a culture of assessment can reduce the stress on your faculty, especially in terms of students. When you educate your students on their role in improving the curriculum and various aspects of the PA program overall, faculty will be relieved to make decisions based on information rather than extrapolating or guessing what the best decision may be. Data collection takes time, but this process needs to be fully implemented and operational before students enter the classroom.