Parliament on Aug. 7 saw Members of Parliament (MPs) dissecting the cause and effects of the major IT outage involving CrowdStrike that shook the world on Jul. 19, 2024 (Singapore time).

The incident affected media outlets, banking services, flights, and transport systems across the world.

Changi Airport, which handles an average of 1,000 flights daily, suffered a notable blow, with airport staff resorting to manual check-ins and flight timings being disrupted.

Minister for Digital Development and Information Josephine Teo came out to clarify the extent of the IT outage.

Most essential services not affected

Most essential services in Singapore were unaffected, according to Teo.

For those businesses that were affected, the impact was mostly on internal staff, while a minority of the cases saw customers impacted by service disruptions.

This included passenger check-ins at Changi Airport Terminal 4 and gantry operations at some HDB car parks.

In a separate written parliamentary reply on Aug. 6, Transport Minister Chee Hong Tat revealed that 108 departing flights were delayed by more than 30 minutes. One departing flight and its turnaround arriving flight were cancelled.

"Fairly innocuous software update"

So what exactly happened on Jul. 19 that resulted in many staring at the "blue screen of death"?

In this particular instance, Teo revealed, it is "not yet fully understood what caused a relatively routine software update" to end up as such.

She would later go on to refer to it as a "fairly innocuous software update" with unforeseen consequences.

Nevertheless, the Ministry of Digital Development and Information (MDDI) has set up an internal task force to engage relevant partners in gaining insights into the incident and assessing whether further measures should be taken to improve Singapore's resilience.

Lessons learned

Considering that not all disruptions can be prevented, Teo said it was all the more important for system owners to be able to recover quickly from unexpected disturbances.

Teo shared that critical information infrastructures, essential services, and government services are all subject to "stringent requirements" and require business continuity plans, disaster recovery plans, and incident response plans.

"When things are running smoothly, businesses may question why they should incur cost or prioritise efforts to assess and improve their resilience measures," Teo acknowledged.

She encouraged businesses to implement precautionary measures before it is too late.

Careful about imposing compulsory requirements

Although she stressed the importance of such measures, Teo also said that the government needs to be "quite careful" when they make such measures compulsory.

She explained that this could take agency and a sense of ownership away from owners of IT systems.

She added that many different components go into a system's resilience.

"To imagine that we have full understanding of all the different things that could cause major disruptions is, I believe, unwise... In the vast majority of the cases, it is important to allow the system's owners and indeed to require the system's owners to take ownership, to build up the system's resilience."

Use of third-party software is "unavoidable"

Despite the recent incident, the use of third-party software is "unavoidable".

Such software offers a "wide range of functionalities" to meet the requirements of various organisations.

"This saves time and resources from having to develop such software from scratch," Teo explained.

She assured that government agencies are required to undergo a "thorough risk assessment" and "mitigating measures" when integrating such software.

These include testing software updates in controlled settings before going live and progressively deploying software changes to small groups of users before rolling them out widely.

Related stories

Top image via MDDI and Markus Spiske/Unsplash