The Quick Take

  • Initiated annual 360 degree evaluations for all members of the department of surgery
  • Developed the evaluation instrument with input from all stakeholders
  • Low scores stimulated rapid improvements in behavior

The Innovation

We began to evaluate individual surgeon performance in terms of the 6 Joint Commission/ACGME axes of performance using 360 degree methodology 3 years ago. At least 80% of our surgeons are employed by the medical center. We initiated the program under a research grant in order to assess the impact on both staff and physicians. Last year, we also used 360 degree evaluation process to evaluate the performance of all Operating Room staff (circulating nurse, scrub techs, anesthesiologists and CRNAs).

The evaluations are sent by email to each rater with a link to the online evaluation instrument. Each person is evaluated by 6-8 raters. For the surgeons, this includes 2 circulating nurses, 2 scrub techs, 2 CRNAs and 2 Anesthesiologist with whom they have worked frequently in the past year. Each surgeon also completes a self- evaluation and evaluates one other surgeon. They must observe a case performed by that surgeon prior to filling out the evaluation form.

We give each surgeon a report comparing the self-evaluation to the scores of the raters on the 6 axes of performance. The Chief of Service is copied on the report. The Chief is responsible to meet with the surgeon to discuss the report, providing positive feedback or coaching to improve performance as indicated. Similarly, each Administrator reviews individual performance reports with their staff.

The evaluation instrument consists of approximately 30 questions. The questions are designed to ask about observable behaviors and to avoid providing judgmental answers whenever possible. A group comprised of the Chiefs of Service of the various OR departments and their Departmental Administrators came up with a draft list of questions. From these, 15 were designated as “must-haves.” The “must-haves” and the other candidates items were then presented to all stakeholders at departmental meetings for discussion. Their votes determined the final form of the evaluation instrument.

The Impact

Our primary purpose in undertaking this mode of evaluation was to help breakdown the hierarchies that are so strongly entrenched in OR cultures. We hypothesized that if surgeons and anesthesiologists were going to be evaluated by all other members of the OR team, then they might show greater respect for them.

Thus far, the program has exceeded our expectations. Most who have scored low responded with rapid improvement in behavior. This was particularly true of those whose scores were unexpectedly low.

Lessons Learned

Although we initially met with resistance as to the appropriateness of the process, the surgeons had to acknowledge the logic.

The first year we used Survey Monkey, but found the data aggregation and reporting was too labor intensive. Now we contract for the service at about $25 per target per year.

Identifying the right evaluators for each target proved to be the biggest challenge. We now subscribe to a data base service (Surgery Compass). With this I can pull out reports that identify the individuals that a particular surgeon has worked with for a specified date range. It automatically sorts the names by case volume from highest to lowest for each of the OR roles. I select the raters manually to make sure that no one staff member is assigned an excessive number of surgeons to evaluate and also to make sure insofar as possible that surgeons are evaluated by different raters each year. I enter this into a spreadsheet which the medical staff office uses to send the e-mails. The program we use to run the 360 degree evaluations allows us to specify the frequency of reminders (weekly x3) and when to close the survey and tabulate the results (a total of 4 weeks).

Contact

Felix Hernandez, MD, MMM

Chairman, Department of Surgery

Eastern Maine Medical Center

Bangor, ME

fhernandez@emh.org

Office: 207.973.8828