Airport meltdown that left thousands stranded ‘was caused by engineer working from home whose password failed’
AN airport meltdown that left thousands of travellers stranded was caused by an engineer working from home whose password failed, an inquiry has found.
Some 700,000 passengers were caught up in chaos during August bank holiday last year when a flight-plan glitch caused the National Air Traffic Services (NATS) computer system to collapse.
Planes were unable to take off and land – causing nightmare delays lasting days.
It also cost airlines £100million in compensation pay-outs.
A Civil Aviation Authority inquiry into the incident today found that IT support engineers were allowed to work from home on one of the busiest days of the year.
The disastrous glitch then struck and was made worse by workers being remote.
The engineer assigned to fix the problem struggled to login remotely because the system had crashed and would not accept his password, it was found.
Engineers then spent an hour-and-a-half making their way to the office where they performed a “full system re-start” – which still didn’t solve the problem.
Four hours after the incident was first flagged, someone phoned the system’s German manufacturer, Frequentis Comsoft, who was able to identify the issue.
As a result, the Civil Aviation Authority today called for senior engineers to be on duty in the NATS offices all times in the hope the problem will not be repeated.
Inquiry chairman Jeff Halliwell said the incident on August 28 represented “a major failure” on the part of the air traffic control system.
He said: “Our report sets out a number of recommendations aimed at improving NATS’ operations and, even more importantly, ways in which the aviation sector as a whole should work together more closely to ensure that, if something like this does ever happen again, passengers are better looked after.
“It caused considerable distress to over 700,000 passengers and resulted in substantial costs to airlines and airports.”
Rob Bishton, chief executive of the Civil Aviation Authority, added: “It is vital that we learn the lessons from any major incident such as this.”
The inquiry found that an automatic flight planning system and its back-up shut down within 20 seconds after a plan for a flight from Los Angeles to Paris (Orly) was received.
This was because of a “unique set of circumstances not previously encountered”, including a pair of duplicate three-letter waypoints, which are used to identify locations.
The system had previously processed more than 15 million flight plans without this scenario being seen.
The failure led to flight plans being manually processed, decreasing the rate from up to 800 per hour to 60 per hour.
A Nats spokesman said: “We would like to apologise again for the inconvenience passengers suffered because of this very unusual technical incident.
“Over the 15 months since this incident, we have worked hard to address the lessons from it, and to ensure it cannot ever happen again.
“Our own internal investigation made 48 recommendations, most of which we have already implemented; these include improving our engagement with our airline and airport customers, our wider contingency and crisis response, and our engineering support processes.
“We fixed the specific issue that caused the problem last year as our first priority and it cannot reoccur.
“We will study the independent review report very carefully for any recommendations we have not already addressed and will support their industry-wide recommendations.”
Desk-shy airport staff working from home were behind flight meltdown in 2023
LAST year, more WFH staff were to blame for a different flight meltdown that affected nearly 750,000 passengers.
Shoddy planning was also to blame for the travel mayhem, according to a review commissioned by the aviation watchdog.
The outage affected nearly 750,000 passengers, including 300,000 people hit by cancellations, 95,000 by long delays of more than three hours, and a further 300,000 by shorter delays.
Downing Street said the scale of disruption was a “completely unacceptable situation” and that the interim report was “concerning”.
The inquiry by the Civil Aviation Authority found fixing the problem was “more protracted than it might otherwise have been” because senior engineers were not in the office over the Bank Holiday.
This is because on public holidays – when maintenance is not routinely scheduled – it was “common practice for staff to be available on standby at remote locations – typically at home”.
But this meant it took one engineer 90 minutes to arrive on site to perform the restart which was not permitted remotely.
The assistance of a more senior engineer was also not sought “for more than three hours after the initial failure”, according to the report.
Ryanair boss Michael O’Leary slammed the National Air Traffic Services engineers for “sitting at home in their pyjamas” on one of the “busiest travel weekends of the year for air travel”.
He added: “In any properly managed ATC service, engineers would be onsite to cover system breakdowns instead of sitting at home unable to log into the system.
“Overpaid Nats chief executive Martin Rolfe’s position in untenable.”