Peer Review & Lessons Learned
In May 2006 US Forest Service Fire and Aviation Management published a briefing paper on Peer Review Process. Later that year, a peer review team used the process to investigate a near miss incident in the Shoshone National Forest and issued a report titled Little Venus Fire Shelter Deployment. This report provides an interesting look at the peer review process and potential benefits of a similar approach to identifying and communicating lessons learned in the structural fire service.
The stated purpose of the peer review process is:
..to reduce errors by correcting or reinforcing upstream behaviors and other factors. Peer reviews provide a means to learn from a variety of situations including close calls, significant events, and other routine performance evaluations. The objective is to create a culture that expects and values peer reviews as an important means to discover subtle indictors of potential future errors and as a catalyst for positive change.
Peer Review and Accident Investigation
Peer review is not limited to investigating accidents and near miss events; it examines organizational performance in a variety of circumstances. However, a peer review and formal accident investigation may run concurrently. As stated in the US Forest Service Peer Review Process Briefing Paper, “this approach helps to segregate human error from intentional disregard of rules and gives the opportunity to identify positive behaviors and decisions even when bad outcomes occur.”
It is important to emphasize that peer review goes well beyond the context of accident and near miss investigation. This process applies to a broader range of significant events.
Key Process Elements
Like entrapment investigation, peer review is a team based process, but the team is comprised of “a small group of operators known for their ability to perform the particular mission in the particular environment, and also known to be insightful, fair, just, and honest”ť. This approach is consistent with the focus of peer review on developing lessons learned.
Key questions addressed in peer review examine individual observations and perceptions and include:
- Action Plan and Leaders Intent
- Situational awareness
- Actions Taken and Not Taken
- Personal Lessons Learned
In many respects the peer review process gathers the same types of information as the National Firefighter Near Miss Program. However, there is a significant difference. In peer review, team members are encouraged to “continue questioning in areas where the reviewers feel disconnect, discomfort, confusion, or curiosity”.
Communicating Lessons Learned
The peer review team develops a report that provides a look from outside the element of the organization involved in the accident, near miss or significant event. This written report identifies the story of the event, reasons the situation developed as it did, and lessons learned. The Peer Review-Purpose and Process Briefing Paper outlines a number of potential benefits:
- Provides feedback on performance and potential areas of improvement
- Assists supervisors in employee development
- Helps guide training strategies, organizational policy, and operating guidelines
- Develops data higher level lessons learned analysis
- Promotes long-term positive shifts in organizational culture
Peer review reports such as the Little Venus Fire Shelter Deployment take a middle ground between a comprehensive organizational assessment seen in some agency reports (see reports from Loudon County Fire and Emergency Management and Prince William County Department of Fire and Rescue) and more limited information provided in National Institute for Occupational Safety and Health (NIOSH) Death in the Line of Duty reports.
Obstacles
Peer review requires a bit of organizational and individual courage and commitment. One element of the deliberate practice required to develop expertise in any field is feedback on results and engaging with that feedback to refine and improve performance. Individuals and organizations must have the courage to ask for feedback and accept performance related feedback, which may be uncomfortable or difficult when things do not go well.
A more fundamental and underlying challenge lies with our underlying assumptions about the nature of fire, firefighting, and the business that we are in. Future posts will address at these important issues.
Ed Hartin, MS, EFO, MIFireE, CFO