Trainee Evaluations, Reviews, and Confidentiality: Gaps for Improvement

Last week, I gave a few personal case studies on what led me to change my mind about full transparency regarding resident progress. The perspective-taking outcome from that has given me a new lens and a different way of viewing this difficult topic.  This week, I want to unpack it a little more.

The ACGME mandates programs to have a Clinical Competency Committee (CCC) with at least three members, one of which is a core faculty member. They are tasked with reviewing each resident twice a year, and after having synthesized their evaluations (formal and informal), they produce a semi-annual review that involves both summative and formative feedback.

The process helps in two ways.

1. It provides a Summative Assessment OF Learning to report to both the PD and the resident, letting them know they are either on track or not for promotion and graduation (i.e., competent to enter practice independently).
2. It provides a Formative Assessment FOR learning for both the faculty and resident to know where the opportunities for growth lie. The way it’s designed is to improve individual residents’/fellows’ progress along a developmental trajectory no matter where they are currently. In other words, it’s as much for growing in excellence as it is for competency. (Within the CCC Guidebook details, the process also implies an opportunity for Formative Assessment AS Learning, i.e., a self-reflected, iterative approach to help develop autodidactic learners access their internal motivation).

As far as I’m aware, most, if not all, programs have written systems for the flow and confidentiality of the information and the due processes of the CCC.

The potential gaps in the process that stand out to me as it relates to our current topic (and questions programs can define for themselves):

1. The integrity of the information post-CCC session. After the CCC meets, there may be curious individuals or stakeholders that were not privy to the information. How often are members reminded of the program’s decisions regarding the disclosure of information? How often are the faculty at large reminded of the policies? How do we sustain the culture to support the policy?

2. Educational Tension. The formative assessment for learning is not solely for the individual resident/fellow for self-directed learning. It also exists to help faculty members educate toward identified growth edges and the degree of oversight for patient safety. And this is the largest crux of the issue as I see it. I don’t pretend that there is an easy (or even a difficult) “one-size fits all” solution. But I offer questions to be thought through.

How do you honor the confidentiality of trainees and highlight areas of focus for the educators? How do you decide which faculty need to know what? Which things, if any, need to be brought to the faculty meeting at large? What things, if any, need to be communicated to other programs/community attendings/non-core faculty? Does the chief or other senior resident need to know? If so, how much do they need to know?

There are a couple of issues here. One involves being objective and specific with each situation. What rotation is coming up? Given the areas of concern, how would each person benefit from being privy to information beforehand? What potential biases or other adverse effects could occur by disclosing information? Who will make the decision, and what is the process?

The second is allowing the trainee to be involved in both the discussion about who needs to know and why, and if they’d like, be present for the conversations or even deliver the information themselves. Transparency and choice are important components of psychological safety - essential in such a vulnerable situation. And this may be the most important detail that is more easily overlooked. What changes can you make to incorporate this viewpoint?

3. Initial Reporting. The CCC is not the entry point for a report, assessment, or evaluation. So even having wonderful processes in place that are spelled out and adhered to will not close all the gaps. Depending on the automated incoming evaluation setup (electronic or paper), the initial reporting may go to the head of the rotation and/or the advisor in addition to the trainee. Informal assessments in the form of email or verbal communications may go to either of those or the attending currently leading the clinical experience, the attending that led the clinical experience at the time, or someone on staff that the reporter is most comfortable relaying information to. In family medicine, this is particularly non-standardized as many other departments and community attendings often report in their own manner. I want to highlight that these informal pieces may be with or without the resident having received the same information. (More often than not, they have not.) Can we adopt a policy that we always encourage the person to also go back and discuss it with the trainee? Knowing many will not, it may help a few occurrances. 

Usually, getting enough evaluations and feedback takes a lot of work. And therefore, informal “hallway conversations” often provide an important role in identifying fundamental areas of concern. 

So, try as we might to standardize the system, there will always be multiple points of entry for the information.

My hope is that programs have a clear, straightforward plan for feedback that comes in. It could look like Faculty A gets “x” information, that information goes to “B”. When there are no red flags, it can go about the usual route to be compiled for the CCC. But when there is a red flag – what’s the process? Does it go to the advisor first or the resident first, or somewhere else? What makes the most sense? How do we make it easily memorable and accessible for any faculty (or other staff)?

To reiterate, unsettling informal feedback is important but creates a vulnerable situation for the resident (and one could argue the receiver of the information as well). Having a transparent, well-thought-out plan in advance is helpful.

I would advocate that the resident be notified first by the original receiver of the information privately (and under extenuating circumstances, told second by their advisor, who was first to be appraised). And it should be explicit what the communication chain has been and will be going forward. Imagine the feeling of being the last to hear a second-hand, unflattering report of yourself that all of your supervisors have seemingly discussed. Let’s avoid that situation.

And yes, when there is some egregious report, multiple parties may need to be notified immediately, but even in that case, the resident should be among the primary.  Let’s avoid meetings about situations where we talk about someone who is even unaware that negative feedback has been received. And, to go back to my case study #5 last week, if an “investigation” is needed, please, by all means, allow your trainee to know what will be taking place in advance. This isn't an episode of NCIS where we need to covertly investigate to 'catch the perp' by surprise.   

4. Communicating the process. I suspect all residency programs have the outline of the CCC review process and all the aspects of due process available for residents in a handbook and/or links to policies and procedures. (Consider adding the handling of the multiple entry points of the informal feedback chain of communication that is decided upon as well). Likely it’s even discussed at orientation, along with 1000’s of other important pieces of information.

Many residents are still unaware of the details of the semi-annual review process. Knowing it takes place and is mandated and done in a fair manner will alleviate some of the discomfort when results are discussed. The regular advisor meeting may be the place to explain the process routinely. “As you may recall from orientation, the program has a formalized process to review all evaluations twice a year at the CCC that is mandated by the ACGME. The members are x, y, and z. They uphold standards for best practices and confidentiality. I, as your advisor, will go over their summary, and you and I will partner to define a plan for your continual progress.” Or something along those lines… Or it could be done by the chief resident in resident or res/fac meeting prior to the routine advisor meetings.

5. I’m a big proponent of checking your own intention before undertaking a feedback session, evaluation, etc. The language by the ACGME is two-fold – both:
   1. Supportive of the educational needs of the trainee: “Improve individual residents’/fellows’ progress along a developmental trajectory,”
   2. Protective for the public: “accountability as medical educators to the public that graduates will provide high quality, safe care to patients while in training, and be well prepared to do so once in practice.”

My personal bias now is always to use the lens of the supportive vantage. When we align with supporting the trainee, we will co-create ILPs for growth (whether it be for competence or excellence), remediation, probation, or extension that, in turn, will automatically protect the public as a secondary outcome. Dismissal is even possible with this lens. Because there are indeed cases in that the resident isn’t progressing despite all your efforts and resources, and what’s in their best interest is not to prolong the inevitable.

I want to highlight the subtlety here between supportive and protective of the public. Protecting has a slightly different stance that can lead to more of a gatekeeping, fault-finding, judgmental, or hypercritical mode. We act differntly from this lens. This is especially slippery when we may not “click” with a resident or when there is a franker interpersonal disconnect. I mean, you and I are human, and we might not really get along with every resident. The shift to staying in a “supportive” approach is subtle. Are you there to find fault or identify and help the growth edges? The gatekeeping will still naturally happen if they indeed are not ready for promotion or graduation. 

To highlight this from an attending standpoint, I offer a recent experience. I was recently greeted in a medical staff office by a sign that said, “We are the first line of patient safety.” It was so interesting, and I understand the reasoning behind it. The credentialing process is essential in keeping bad or fake doctors out of hospital systems. However, when that is what’s highlighted in the reception area, it sets a different tone. I immediately felt uncomfortable – and I have nothing to feel guilty about! It’s much different from my previous work with a hospital med staff office that had a mission statement about supporting the medical staff to help them excel in patient care. This is the subtle shift I’m highlighting.

Along with these thoughts is a close parallel. That of bias. We all have both conscious and implicit biases. I appreciate the language by the ACGME (and the multiple supportive studies they cite) that we need to “Anticipate biases on the part of both oneself and committee members, and intentionally cultivate greater insight on biases and strategies to mitigate them.” (More to come on this topic later this month.)

Wrapping up my thoughts on today's overview of the intentions, processes, and communication of those processes to help support residents in the face of concerning feedback, I have one last take.

6. Resident Advocate. It is common to need to bring in the head of the CCC, an APD, a BH faculty, and/or the PD to meet when there is a pattern of concerning feedback, failure to progress, or egregious reports.

How this is handled can also improve in some instances. Having the advisor forewarn the resident that others will be attending a meeting is very important. And they need to have at least an honest opportunity for an advocate. Most programs default to that being the advisor. And, since “advocate” is one of the roles of an advisor, that makes sense.

Two recommendations: 1. If an advisor is internally conflicted, they need to be able to express that they don’t feel they will be in the meeting fully as an advocate to allow the resident to bring someone else (and perhaps maybe consider switching advisors). 2. Even if the advisor feels comfortable in the role of advocate, the resident may not have the same level of trust. We should make it easy for them to ask and allow – “Dr X, your advisor will be plan on serving as your advocate in the meeting, unless you would be more comfortable with someone else in that role…” or even more plainly, "Who would you like to invite to attend as your advocate?"

That’s a lot for today – thank you for sticking with it. I’d love to hear your perspectives, experiences, and ideas. Message or email me. Next week, we will review a bit about psychological safety. 

Until then, Have a joy-filled week!  Tonya

If you have a resident in difficulty that would want to opt-in to coaching, I'll be happy to work with them as a safe, confidential thought partner to help evaluate the feedback, gain insight, formulate plans and navigate the emotions that come up in such difficult times. Learn more HERE