PIFmanship 101 (or how to survive writing/reviewing program information forms)

PIFmanship 101(or how to survive writing/reviewingprogram information forms) Department of Graduate Medical Education Stanford University Medical Center

Session Objectives By the end of this session, you will be able to… • Understand the basics of putting together a program information form (PIF) • Improve outcomes by avoiding many of the PIFmanship “pitfalls” that lead to RRC citations • Save time when preparing/ reviewing PIFs for internal reviews and site visits • Decrease the fear and stress associated with completing a PIF

“I see a Site Visit coming in your future…”

Gentle words of wisdom… Your PIF is the site visitor’s first impression of your program, so... • Be accurate • Answer the question that’s asked • Be terse/tight • Have documentation to back up your answers • Start early – it takes months to write a good PIF • This is not something your mother, spouse, best friend, or admin asst. can do for you!

Common vs. Specialty There are actually 2 PIFs to complete: • The Common PIF • Addresses the program’s compliance with the Common Program Requirements (common to all GME programs) • The common PIF is completed online via the ACGME Web Accreditation Data System (WebADS) • The Specialty PIF • Addresses compliance with the specialty-specific program requirements • Depending on specialty, may be a Word document downloaded from the ACGME Site to be completed offline, or may be completed online via the ACGME Web Accreditation Data System (WebADS).

Common PIF • The common PIF contains questions regarding participating sites, faculty CVs, evaluation, and duty hours. • Most of the data should be updated annually by your coordinator. • To update the Common PIF: • Go to: www.acgme.org/ads • Use your ACGME assigned User ID & Password • Go to the PIF preparation section (left-side menu) • Once all of the data is entered and VALIDATED • Print as PDF

Common PIF – Page 1

Common PIF – Page 1 • Data is auto-populated based on entries elsewhere in WebADS • Original signatures required: • Program Director • Core Program Director (subspecialty programs only) • DIO PIF PIF

Common PIF – Page 2 • Previous citations are auto-populated • Remember, your citations and corrective actions live forever – respond with care. WebADS

Common PIF – Page 2 • “Major Changes” include: • Changes in program format • e.g., Have you gone from 3 to 4 years? • Changes in resident complement? • Changes in program leadership? • Changes in participating sites? • Only include changes since the last site visit!

Common PIF – Page 3-4 • Sponsoring Institution • Auto-populates from WebADS • Single/Limited Site Sponsor - only sponsors one program • Participating Sites • Only list sites that provide a required one month full-time equivalent educational experience • Required means all residents rotate to that site • Make sure rotation lengths add up to 12 months per year across all sites

Common PIF – Page 3-4 • Participating Sites • Brief Educational Rationale • Residents are exposed to a wide array of patients with advanced diseases, larger indigent population, and many minority groups not as well represented in the other participating sites. There is an excellent exposure to the primary components of general surgery especially trauma and surgical critical care.

Common PIF – Page 3-4 • Participating Sites • Program Letter of Agreement (PLA) is required unless under governance of sponsor! • We’ve posted a PLA template to our GME web site • Note that the PLA is an agreement between the Program Director and the individual at the participating site charged with oversight of the residency program. As such, the PLA may be between you and yourself… • e.g., at Stanford, the PM&R Program Director is also the site director at the VA hospital

Common PIF – Page 5 • Program Director MUST be able to approve the selection of teaching faculty • The Program Director MUST evaluate and approve continuation of teaching faculty • Program Director MUST comply with the university’s or medical center’s policies on selection, evaluation, promotion, disciplinary action and supervision • Refer to the House Staff Policies and Procedures document posted on our GME web site: http://med.stanford.edu/gme/policy/ • Program Director MUST comply with ACGME and RC policies and procedures • Institutional • Common • Specialty-Specific

Common PIF – Page 5 • Physician Faculty Roster • List Alphabetically and by Site • Faculty who spend at least 10 hrs per week in resident education • Hours/week devoted to education should be realistic! • Board certification • If double boarded include both boards • If sub-specialty program director or faculty, ensure primary boards are included • Make sure the roster data matches CV data

Common PIF – Page 6 • Faculty CVs • Great potential for a HUGE number of citations • All fields must be completed (NO BLANKS) • Accurately list training history including GME • Ensure certification is valid • Ensure licensure is current and has not lapsed

Common PIF – Page 6 • Faculty CVs • Selected Bibliography, Review Articles, & Activities • Strict limit of 10 • “…from the last 5 years” excludes any before 2006! • Publications should not be “in press” if submitted many years ago… • If not ABMS certified, explain…

Common PIF – Page 6 • Be concise, but not THAT concise. Try instead: • <Program Director> oversees the operations of the entire program; supervises trainees during patient-care activities; mentors trainees’ research projects; leads lectures and seminars; monitors duty hour compliance; and coordinates evaluation of courses, rotations, trainees and faculty.

Common PIF – Page 7 • Non-Physician Roster and CVs • Again, accurately complete all fields and • Observe the 10 item / 5 year limit

Common PIF – Page 8 • Number of Positions • Note: If you have a resident making up a maternity leave, you must ask ACGME for an extra slot if you are over your quota • Actively Enrolled Residents • Other than interns, everyone should have prior years of GME • Program Director MUST obtain summative evaluation of previous experience for transfers

Common PIF – Page 10 • Transferred, Withdrawn or Dismissed Residents • Residents who resign are NOT dismissed • Must provide competency-based summative evaluation to new program for transfers • Evaluation • YES, residents are evaluated following each learning experience (i.e. rotation) • YES, evaluations are documented (state how) • Electronically • Resident Files

Common PIF – Page 10 • Methods of Evaluation • Assessment Method • Direct Observation • Simulation • OSCE • Etc • Evaluator(s) • Program Director • Faculty • “360” (required as of 2007) • Nurses • Ancillary Staff • Patients • Other

Common PIF – Page 11 • Answer the question that’s asked!

Common PIF – Page 11 • Describe how evaluators are educated to use the assessment methods listed above so that residents are evaluated fairly and consistently. • The Program Director meets with evaluators annually before new trainees start the program to review and discuss the core competencies, competency-based performance evaluations, and assessment methods to be used. The electronic assessment system and the rating scales are also reviewed and discussed during faculty meetings in order to ensure that evaluators are fully educated and up to date with the assessment methods and processes.

Common PIF – Page 11 • Describe how residents are informed of the performance criteria on which they will be evaluated. • At the beginning of each academic year, the Program Director conducts an orientation to address several key issues related to the residency including the performance criteria on which the residents will be evaluated. During this orientation session, the Program Director carefully details the specific evaluation methods to be used. Both the criteria and methods are also documented in the residency handbook. Additionally, at the beginning of each block rotation, the rotation director meets with the resident and carefully delineates the expectations and performance criteria on which the resident will be evaluated for that specific rotation.

Common PIF – Page 11 • Describe the system to ensure that faculty complete written evaluations of residents in a timely manner following each rotation or educational experience. • At the end of each block rotation, our online resident data collection and tracking system sends an automated reminder to the service attending(s) to evaluate the resident(s) on that particular rotation. Using this online system, the Program Coordinator tracks pending evaluations and follows up with faculty as needed to urge them to complete their evaluations on time. In the unlikely event that a faculty member is unresponsive to the coordinator's requests to complete evaluations, the Program Director contacts the faculty member and requests him/her to complete the evaluations.

Common PIF – Page 11 • Describe the process used to complete and document written semiannual resident evaluations, including the mechanism for reviewing results (e.g., who meets with the residents and how the results are documented in resident files). • Resident performance is evaluated by the teaching faculty at the conclusion of each rotation using an electronic evaluation form. Additionally, each faculty mentor meets with their resident advisee quarterly and documents a summary of the meeting to be placed in the resident’s file. The Program Director meets with each resident on an individual basis semiannually to review the accumulated performance evaluations and mentor notes, provide feedback, and update the resident’s learning plan as appropriate. A summary of these meetings is documented and placed in the resident’s file. The residents are free to review the contents of their records at any time.

Common PIF – Page 11 • Describe the system used by the residents to provide annual confidential written evaluations of the teaching faculty (have examples and forms available for review by site visitor). • Residents annually confidentially evaluate the teaching faculty using <system>. The electronically submitted evaluation forms are anonymous. Residents are sent e-mail reminders about completing evaluations in a timely manner by the Fellowship Coordinator. Paper copies of completed evaluations without personal identifiers are printed by the Program Coordinator, reviewed by the Program Director, and presented to individual faculty members for review and consideration.

Common PIF – Page 11 • Describe the program's (or Department's, if applicable) system for evaluating and providing feedback to the teaching faculty. • Residents annually confidentially evaluate the teaching faculty using <system>. The electronically submitted evaluation forms are anonymous. Residents are sent e-mail reminders about completing evaluations in a timely manner by the Fellowship Coordinator. Paper copies of completed evaluations without personal identifiers are printed by the Program Coordinator, reviewed by the Program Director, and presented to individual faculty members for review and consideration. If and when an evaluation reveals an issue with a particular member of the teaching faculty, the Program Director (and/or the Department Chair) meets with that faculty member more urgently to address the issue. Additionally, the Program Director meets with all faculty on an annual basis to review resident feedback and implement any necessary changes.

Common PIF – Page 11 • Describe the approach used for program evaluation. • Residents and faculty annually evaluate the program using the <system>. The electronically submitted evaluation forms are anonymous. Residents and faculty are also encouraged to provide feedback to the Program Director whenever any issue arises or as they see opportunities for improvement. Additionally, residents and faculty participate in an Annual Program Review Meeting led by the Program Director. Aggregated data including the most recent ACGME survey results and the resident/faculty program evaluation results are reviewed and used at this meeting to improve the program. The Program Coordinator keeps minutes during this annual meeting and documents any plans to address areas requiring improvement. Action plan progress is monitored and documented by the Program Director.

Common PIF – Page 11 • Describe one example how the program used the aggregated results of residents' performance and/or other program evaluation results to improve the program (have the written plan of action available for review by the site visitor). • Over the past two years, the evaluations by the faculty, as well as the results of the didactic EMG examinations, and resident performance on the SAE examination have all reflected an average to less than average performance by the residents in the area of electrodiagnosis. The educational committee discussed this area of concern, going over all of the evaluations, reviewing the examinations, and determined an action plan. Each of the participating sites exposing the residents to EMG increased the number of didactic sessions in this area, especially for the first-year residents. Noon conferences were instituted at Valley Medical Center and the VA emphasizing EMG education, which included hands-on sessions conducted by the faculty, with senior resident contribution. The EMG portion of the didactic curriculum was moved forward toward the beginning of the academic year (September) to expose the junior residents to the electrodiagnostic educational material earlier in their residency. The anatomy lab was also restructured to supplement the functional anatomy and EMG correlative didactics.

Common PIF – Page 11 • Describe the improvement efforts currently undertaken based on feedback from the ACGME Resident Survey. • Program Director MUST: • review the results of the ACGME Resident Survey each year • discuss the results at your Annual Program Review Meeting • address each area of concern (more than 20%) or any negative Duty Hour response • While smaller programs do not receive their results annually, results aggregated over several years may be accessible – check in WebADS and/or with your DIO

Common PIF – Page 12 • Resident Duty Hours • Use the summary report generated by your tracking system

Common PIF – Page 12 • Briefly describe how the faculty provides appropriate supervision of residents in patient care activities. • While the supervision of residents in the program is designed to provide gradually increased responsibility and maturity in the performance of the skills, all residents are supervised at all times. The required Level of Supervision for specific tasks is assigned based on level of training unless by exception, the Program Director indicates that further training is required before such approval is granted for a given resident. The Levels of Supervision provided by faculty are defined as follows: • Level-1: Direct Supervision - The supervising physician is physically present with the resident and patient • Level-2: Indirect Supervision - • A: Direct supervision immediately available – The supervising physician is physically within the confines of the site of patient care, and immediately available to provide Direct Supervision • B: Direct supervision available – The supervising physician is not physically present within the confines of the site of patient care, is immediately available via phone, and is available to provide Direct Supervision • Level-3: Oversight – The supervising physician is available to provide review of procedures/encounters with feedback provided after care is delivered • Residents are responsible for knowing the limits of their scope of authority. Whenever a question arises about resident competency to perform a procedure independently, the attending physician is consulted.

Specialty PIF • The specialty PIF is generally a Word document* and found on the ACGME web site: http://www.acgme.org/acWebsite/navPages/nav_comPIF.asp • The specialty PIF contains questions regarding the ACGME general competencies and may request a block diagram for your program, a narrative description of your program, documentation of scholarly activity, and/or case logs. * For some programs, such as Medicine Specialties and Ophthalmology, the Specialty PIF is accessed and completed via WebADS, just like the Common PIF.

Specialty PIF – PBLI • Describe one learning activity in which residents engage to identify strengths, deficiencies, and limits in their knowledge and expertise (self-reflection and self-assessment); set learning and improvement goals; identify and perform appropriate learning activities to achieve self-identified goals (life-long learning). • Describe one learning activity. • e.g., programs may use a structured process for reflection in which a faculty advisor guides the resident in using feedback , evaluations, and/or in-training exam scores to inform the self-assessment process. • Documentation of the semi-annual evaluation meetings in which this process is demonstrated would provide evidence that this requirement is being addressed.

Specialty PIF – PBLI • Describe one example of a learning activity in which residents engage to develop the skills needed to use information technology to locate, appraise, and assimilate evidence from scientific studies and apply it to their patients’ health problems. The description should include: a) locating information, b) using information technology, c) appraising information, d) assimilating evidence information (from scientific studies), e) applying information to patient care. • Describe one example. • An appropriate learning activity could be a structured activity such as a journal club presentation, critical appraisal of a topic, or educational prescription with appropriate faculty oversight and formal assessment of skills. • Additional documentation would be the written goals and objectives for this learning activity and how residents are assessed.

Specialty PIF – Professionalism • Describe at least one learning activity, other than lecture, by which residents develop a commitment to carrying out professional responsibilities and an adherence to ethical principles. • At intern retreat we devote an afternoon to discussing the role of a pediatrician in the long term care of a chronically ill and ultimately terminally ill child. We use the film “Cameron’s Arc” to initiate a discussion of the crucial role the pediatrician played in the care of a child with Tay Sachs disease. Focusing not on the disease per se, but on the role of the pediatrician, we identify the important character traits that made the pediatrician so very crucial to this family and to this child: we discuss the sense of ownership, the process of bringing a family to an understanding of the disease they face, the essential simultaneous expression of empathy and control, the style of communication that adapted to each parent’s needs, the choice of timing of various conversations as the condition and circumstances change for this child and her family, and the holistic approach to a disease process that requires direction of difficult decisions and participation adjusted to the needs of a family at various points in the journey a disease may dictate. At the end of the exercise, we share and reflect as a group on what we each found so essential to the “meaning of being a doctor” that was so well portrayed by the pediatrician in this film.

PIFmanship 101 (or how to survive writing/reviewing program information forms)