Upset Recovery Performance Investigation by APS

Preamble to Investigation Results

GAJSC Loss of Control In-Flight Statistics 2011

Loss of Control In-Flight (LOC-I) is the leading cause of fatalities in both commercial and general aviation over the past 10 years. Training department managers and flight departments seeking to mitigate lethal threats to the safety of the traveling public, clients and crews must necessarily address LOC-I. Although academic training alone can have value, the unique challenge of mitigating LOC-I is its occurrence in regimes of flight that are not comprehensively addressed in traditional licensing training and requires practical hands-on training. Certain physiological and psychological aspects of  LOC-I events point to the powerful value of training in real aerobatic-capable airplane. This training must occur within a building block curriculum under the guidance of specialized upset recovery instructors to establish pilot processes and disciplines to overcome the reality of an event beyond the pilot's experience if those pilots have not been indoctrinated into comprehensive upset prevention and recovery training during their training.

CAST LOC-I Statistics 2012

In a report issued by Boeing in July 2012 (to the right), LOC-I represents the most severe cause factor in commercial aviation over the past 10 years, resulting in the most crash-related fatalities from 2002 through 2011 – even more than CFIT. According to the Commercial Aviation Safety Team (CAST), there has been recent aviation-industry emphasis on fatalities being a more accurate representation of the severity of an accident as opposed to hull-losses that has historically been used as an indicator of crash severity. Aviation safety organizations and legislating agencies continue to accurately identify the lethality and severity of LOC-I.

 The Investigation of LOC-I Mitigation

Cover Story: Flightglobal Nov 2012

In late 2007, Aviation Performance Solutions (APS) initiated an investigation to measure pre-training and post-training pilot skill development in relation to APS-specific Upset Prevention & Recovery Training processes. The data gathering of the investigation occurred over a three month period (Dec 2007 through February 2008) and included 115 pilots. All pilots trained at APS (without discrimination based on experience, flight hours, airplane type or ratings) were assessed in order to provide an opportunity to breakdown performance into a variety of demograhic filters using data analysis technology after the study was completed. One filter result, the Professional Pilot Filter, is shown later in this posting under the title 'Filtering the Above Study for Professional Pilots' where more details are provided on the specific scenarios used in the investigation.

APS investigation participants (the pilots being assessed) completed questionnaires prior to commencing flight training and then again after the training was completed. All pilots being assessed participated in a 2-hour ground briefing on general Loss of Control In-Flight concepts and mitigation techniques and were made aware that their initial performance skills would be evaluated in both their first and last flights during their APS training experience. Following stringent published performance parameters, APS Instructor Pilots evaluated candidates during both their first and last flights to assess competence in recovering from five key upset recovery scenarios. The pilot tracked these results by filling out the data form immediately after the completion of the evaluation flight. The results were entered into a data tracking resource for later analysis.

Overall Results Summary

The overall test group results are shown below and do not include data from the 35 pilots in the overall test group that were returning to APS for recurrent training during the 3-month period of the study.

Ability to Successfully Recover

• Before APS Training: 28.1%
• After APS Training: 96.3%
• Retention of Skill (Average of 19-months after Initial Training): 86.7%

Anecdotal Observations:

1. Why did professional pilots seem to do better during initial testing? Of interest in the study, particularly in relation to the Professional Pilot Filter below, was the apparent ability of Professional Pilots to perform better in this investigation as compared to the overall pilot population although neither group demonstrated a capability over 50% during initial testing. This was not something that had been readily apparent to APS prior to the study and resulted in speculation on how this type of result may have occurred. The most popular opinion among the APS team was that Professional Pilots were generally better professional students and were able to more effectively apply the theoretical knowledge gained in the 2-hour preparatory briefing during the practical training session. SEE NOTE at the end of this posting under the section title 'Update: 2010 Enhanced Data Analysis' for more on this particular point based on data analysis as opposed to speculation.
2. Why did these same professional pilots not retain the training as well? Similarly, our team was surprised that this same group tended to retain their skills less comprehensively than the overall pilot group. After being away from the training for an average of 19-months, the Professional Pilots demonstrated a retention of 76.4% whereas the overall pilot group averaged a retention of 86.7%. This latter result was anecdotally attributed to the professional pilot being exposed to elements of negative training during their type and recurrency training such as the then-prevalent focus on minimizing altitude loss as a first priority in the stall (see AC 120-109 for industry enhancements to that inappropriate philosophy).
3. In the real world, is this the pilot performance we should expect? Due to the preparatory 2-hour training session and pilot fore-knowledge of having their skills being evaluated, the APS team feels strongly that the real day-to-day capability of pilots to be aware of, prevent and recover from a developing loss of control in-flight condition is likely much less favorable than the initial skills results presented in this investigation summary.

On-Going Research on this Initial APS Investigation

As an internal investigation, the results and analysis in the study have limited industry application as it was completed internally using APS resources, technology and personnel. A team of independent PhD researchers is currently working with the APS team to verify and enhance the validity of the these results in a 2-year study. This formal research will include both the framework of the above APS investigation but is also assessing the transfer of skills acquired in a light aerobatic airplane (using techniques and methods proprietary to APS) into transport category airplanes. It is anticipated this latter formal research project will be publishing its results in late 2013.

Filtering the Above Study for Professional Pilots

PROFESSIONAL PILOT FILTER AS FOLLOWS:

Data Filtered Below Includes All Pilots Meeting the Following Criteria:

• Pilots Flying Turbo Prop and/or Turbo Jet Aircraft (total of 75 pilots)

Group Demographic of Professional Pilots Evaluated (Including Initial and Recurrent Participants):

• 88.0 % had greater than 1500 hours of flight experience
• 91.6 % were between the ages of 25 and 59 years of age
• 51.4 % were certified flight instructors
• 81.3 % had less than 10 hours of aerobatic experience

PERFORMANCE EVALUATION RESULTS

Participant Recovery Performance Evaluation for Initial Courses (Before Training versus After Training):

• Training Courses in the study averaged out to 4.4 training missions per course
• Retention of Skill
  Important Note: Of the overall test group of 115 pilots, 35 pilots were repeat customers attending a recurrent upset recovery course at APS. Recurrent participants demonstrated 76.4% retention of skill returning after an average of 19 months between Initial and Recurrent Training programs. Skills are expected to atrophy at a greater rate the longer pilots delay time between Recurrent training courses.

PARTICIPANT FEEDBACK - BY PROFESSIONAL PILOTS AFTER COMPLETING TRAINING

Course Components were Evaluated as Follows:

VALUE TO PILOTS:

100% of the participants indicated that LOC-I training as provided by APS was valuable to all pilots with 64.0 % of those votes indicating that APS LOC-I training should be mandatory in pilot certification.

SKILL AND KNOWLEDGE: 100% indicated they learned quite a bit and developed life-saving skills with 76.0% of those votes indicating their understanding and pilot skill-set related to upset recovery training had grown dramatically

AIRCRAFT: 89.3% evaluated the Extra 300L training aircraft as being EXCELLENT with the remainder of the votes indicating it was ABOVE AVERAGE

FACILITY: 73.3% evaluated the APS facility as EXCELLENT with an additional 25.0% assessing the APS Facility as ABOVE AVERAGE

OVERALL EXPERIENCE: 96.0% of the participants evaluated the overall experience as EXCELLENT with the remainder indicating it was ABOVE AVERAGE

MANUAL: 68.0% of participants rated the APS Training Manual as EXCELLENT with an additional 26.7% ranking the manual as ABOVE AVERAGE

SIMULATOR TRAINING: 100% of participants indicated that upset recovery training in a real aircraft develops life-saving piloting skills and awareness that cannot be taught in a simulator. Participants identified the following factors as the primary critical training areas that cannot be duplicated in the simulator:

APS UPSET PREVENTION & RECOVERY TRAINING AS A MITIGATION TO LOSS-OF-CONTROL IN-FLIGHT

100% of participants indicated that the solution to dramatically reducing the risk of Loss-of-Control In-Flight must include specialized upset recovery training in real aircraft as provided by APS Emergency Maneuver Training. 61.8% of those votes indicated a full solution should also include extensively updated and redesigned simulator training profiles and curricula.

Update: 2010 Enhanced Data Analysis

In 2010, improved analytical tools applied to the gathered data during the study revealed that pilots with unusual aerobatic experience (more than 10-hours) attributed to a higher density of the assessed data in group defined by the Professional Pilot Filter. When the data is filtered to accommodate for the density of more experience in aerobatics, the disparity between the Professional Pilot Filter group and the Overall Group reduced significantly but does show the same trend of Professional Pilots performing slightly better during initial testing and retaining slightly less during recurrent training in comparison to the overall study group (See Anecdotal Observations in the Overall Results Summary section above for more specifics related to this statement).

Although we have not published the updated results that considers the 2010 enhanced data analysis described above, the result of filtering out pilots with more than 10-hours of aerobatic experience in the overall investigation results shows the initial overall pre-training 'Ability to Successfully Recover' skills of pilots to be even worse, a range of 16.8% to 24.3% depending upon which demographic of pilots are being filtered. Since the results are worse in a situation where pilot performance is bad enough already, we didn't feel this distinction would significantly alter the message of urgency especially given that a formal research project is now underway. At the end of the day, it is our preference at APS to seek formal professional research results (see above section: On-Going Research on this Initial APS Investigation) that will have far-reaching industry value than for the APS team to make continued efforts to re-analyse our internal investigative data.

Be Sociable, Share!

•