PA Program Assessment Process
The leadership and faculty of the MS in PA program at CSE are dedicated to a process of continuous self-evaluation and analysis of all aspects of program operations. The faculty executive committee (FEC) and the curriculum committee (CCOM), which are briefly described below, determine all program decision-making.
FEC: Faculty Executive Committee
The FEC is comprised of all faculty and staff, chaired by the program director. This committee oversees and votes regarding all programmatic decisions, both assessment and operational. This committee will receive recommendations from the curriculum committee (CCOM). All programmatic decisions involving curricular modifications, implementation of policy, and programmatic self-assessment and accreditation, are voted on and enacted by this committee.
CCOM: Curriculum Committee
The CCOM will review all data received from the assessment process. The curriculum committee analyzes data generated from the programmatic self-assessment process and makes recommendations for improvement to the FEC for consideration.
The MS in PA program curricular plan evaluates all aspects of the curriculum, including formative and summative outcomes. The curriculum will be evaluated regularly by achieving educational best practices, determining compliance with ARC-PA standards, and measuring graduates’ results on the NCCPA examination. Program alumni and employers of these alumni will be surveyed to determine if the program is achieving program goals, as well as mission and vision statements.
The Curricular Strategic Plan is implemented through regular faculty meeting and a yearly Annual Program Review meeting to comprehensively evaluate all aspects of the program. The loop of programmatic improvement represents a critical, self-evaluative and analytical process that results in appropriate changes in the curriculum as needed to ensure the mission is achieved. Curricular alignment with ARC-PA standards, NCCPA blueprint, OSCE scores and survey results contribute to this planning process. The MSPA program is therefore a living organism impacted by these aforementioned assessment processes with appropriate modifications and programmatic improvements based upon valid data sources. In order to keep current with the ever-changing paradigm of PA practice, a culture of continuous self-analysis will create an ethos of academic excellence among the faculty and students.
The programmatic assessment process also includes ongoing strategic planning. This process is aligned with university strategic planning and is driven by the assessment process. The curricular strategic plan is closely aligned with the operational strategic plan and is supportive of personnel, financial and physical resources. The cycle is complete as the strategic plan and corresponding changes reflect the mission of the program.
Student learning outcomes will begin with all principal and instructional faculty members being involved in the evaluation of students. During the didactic phase of study (academic year), students are evaluated by course instructors who are principal faculty, instructional faculty or both. The faculty member who was assigned as the course master (main faculty member designee) for that course evaluates all courses taught by principal faculty. For courses that are team taught, principal faculty will provide questions to the course director for inclusion into assessment instruments.
The MS in PA program also assesses students following each applicable rotation in the clinical-year phase of the program. The clinical team under the direction of the Director of clinical education will be responsible for organizing callback days and collecting data on clinical aspects of the program. The data collected will encompass PAEA End of Rotation™ exam (EORE) scores, preceptor evaluations, oral presentations required electronic entries, as well as periodic system-focused patient exams. This data will be used to track student progress and overall program achievement of competencies. The EOREs are all-encompassing exams that cover material graduates of the CSE PA program are expected to know for clinical practice and to perform satisfactorily on the PANCE. The EOREs will provide comparison of CSE PA students to other PA students across the nation.
Students will also be evaluated on final evaluation from the preceptor attending and participation at professional seminars on callback days, clinical rotation specific documents/assignments, submission of student evaluation of preceptor and site and successful completion of required electronic entries.
The electronic tracking system used by students during the clinical year will be used for assessment purposes. The collected data will be reviewed and assessed by principal faculty, as well as the coordinator of research and assessment.
The program faculty realizes that aspiring to successfully measure learning outcomes in the clinical year does not merely mean a specific number of patient encounters have been logged by a student. Rather than just relying on numerical evidence of case logs, the program is developing a system of measuring clinical experiences that demonstrate that students are growing in clinical acumen as the clinical year progresses.
The mission of the program focuses on special needs populations therefore students will be expected to demonstrate higher levels of patient interactions as they grow in competence as clinicians. The program will aspire to elevate the assessment process in the clinical year to successfully grow from novice to skilled learner. The program faculty will define program expectations and competencies to be acquired during SCPEs and attempt to connect this process with the didactic curriculum. They believe that the curriculum is a continuum from day one to graduation will guide the program toward a successful integrated assessment system.
During each semester, students will also be graded on complete system-focused, problem-specific simulated patient encounters. These exams will be organized and scheduled by assigned academic-year and clinical-year principal faculty members. These exams will be in preparation for the end-of-the-year objective standardized clinical examination (OSCE), which must be passed in order to matriculate to the clinical phase of the program and to fulfill a graduation requirement.
The program will conduct and document a summative evaluation of each student during the clinical year of the program with the final SUMMATIVE evaluation within the final three months of the program to help verify each student is prepared and competent to enter clinical practice. The summative evaluation will include assessment of student knowledge, interpersonal skills, patient care skills and professionalism. Part of the summative evaluation process involves cognitive assessment to provide feedback on strengths and weaknesses for both students and the program. The cognitive evaluation is comprised of multiple parts. During the first two semesters of the clinical year, students will take comprehensive examinations (SUMMATIVE I comprised of 360 questions that follows the National Commission on Certification of Physician Assistants (NCCPA) blueprint and mimics the PANCE in areas of length, content of task. The students will then complete a 700-question comprehensive examination (SUMMATIVE II) within the last four months of the program. SUMMATIVE II will also involve an OSCE. The OSCE will include multiple stations to assess psychomotor skills, history taking skills, physical examination skills, interpersonal assessment and interpretation of laboratory data. College of St. Elizabeth’s School of Medicine administers the practical exam component. The student will be required to generate a primary diagnosis, differential diagnoses and plan, as well providing appropriate patient education. The OSCE will be administered with sufficient time that allows students to be remediated within a specific domain(s) in which their performance was below the satisfactory cut score. The program will also use a summative professionalism assessment tool as a component of the SUMMATIVE evaluation processes; however, evaluation of professionalism will also be conducted at the conclusion of each semester during the program. Therefore, professionalism will be assessed both as a formative and summative process. This will enable faculty to provide feedback about student’s professional skills and provide remediation as needed. The preceptor evaluation also provides a composite data point for professional behaviors.
In addition to student evaluation, assessment of the program will also occur throughout both academic and clinical years, as per ARC-PA standards. If at any time, the program is determined not in compliance with its accrediting body standards, it will be documented and a plan will be devised, implemented and assessed; thereby, closing the feedback loop. The program Director with 26-years of experience will oversee and guide this process to ensure compliance with the ARC-PA standards regarding all aspect of program assessment.
Principal and adjunct faculty, as well as the program director, will actively participate in the evaluation of the MS in PA program. Other instructional faculty including preceptors as well as the t will play a role in continued program assessment. Evaluation of the program will be continuous and comprehensive and will be discussed at faculty meetings and at the annual faculty retreat.
The on-going assessment process will be initiated for the inaugural class and maintained throughout the program. At each annual faculty retreat, the ongoing development and maintenance of curricular standards will be benchmarked against program competencies, ARC-PA standards and NCCPA blueprint topics. Measurement of whether the program is meeting the learning outcomes and goals will be embedded into the annual review.
Outcomes generated from the previous year will include course-embedded assessments and overall course grades, student evaluation of faculty/courses, cumulative program GPAs, OSCE performances, Physician Assistant Clinical Knowledge Rating and Assessment Tool (PACKRAT) evaluations, SUMM I/SUMM II , preceptor evaluations of cognitive, psychomotor, and professional behaviors, student retention and graduation rates, graduate surveys, exit surveys, graduate scores on the PANCE, job placement rates, employer surveys and faculty and preceptor retention. Data collected annually in these areas will be analyzed for any weaknesses and will be triangulated analyze for emerging trends. This result in necessary adjustments to the curriculum, policy and procedure, and pedagogical methods. Programmatic changes are incorporated into the upcoming year as needed.