Why The Uk Public Sector Still Creaks Along On Cobol

Feature The UK government has gone all-in on AI. More than 50 years after Harold Wilson gave his famous “White heat of technology” speech, this is the hot new thing. An AI Strategy has been released. Datacenters are planned. Steps to strengthen AI supply chains are being formulated. And of course, the public sector will lead by example in AI usage.

Whitehall might talk a big game when it comes to IT, but so far the words haven’t fit the picture. Its own State of Digital Government report, released in January, found the central government’s IT portfolio wanting. It classified 28 percent of central government systems as legacy, rising to 70 percent in some areas. HMRC is still paying contractors top dollar to maintain COBOL systems.

Legacy has a specific meaning in government, per the study: “unsupported, has known vulnerabilities, is difficult to maintain, or is unable to meet current business or user needs.”

“For years this problem has plagued both public and private sector organisations,” says Heather Cover-Kus, head of central government at UK tech industry association techUK. “However, the issue is more complex for the public sector as the cost of getting things wrong is very high, as is the cost of doing nothing.”

The Post Office Horizon scandal is a case in point. Fujitsu-ICL’s 1999 system performed so poorly that postmasters were routinely and falsely accused of theft, driving several to suicide. That system is still operating, and although it was supposed to be replaced by 2025, a May 2025 tender says it could now run until 2033.

Why aren’t we upgrading these systems more quickly? “It’s always going to be more difficult to get senior leaders to focus on the plumbing,” warns policy analyst Gavin Freeguard, former program director at the Institute for Government, where he led its work on digital government, data, and transparency.

Fixing broken IT is simply “not as sexy” as AI, he says. “There’s a real danger at the moment of people thinking AI can just paper over the cracks, when actually you’ve got to fix the cracks before you can use AI effectively.”

How did we get here?

The roots of these legacy systems go back decades. Successive IT policies have settled atop each other like political silt.

“This means that organizations’ struggles with technology are commonly rooted in highly complex, brittle systems which encode arcane policy; difficult to maintain or transform,” says Peter Chamberlin, senior director, technology at transformation consultancy Public Digital.

Many of the current issues have their roots in the outsourcing trend of the nineties, explains Helen Margetts, professor of society and the internet and professorial fellow at Oxford University’s Mansfield College.

“You have to look back in time to understand the way government’s IT capability was originally built up and then effectively privatized through public private partnerships and these very, very large outsourcing contracts with systems integrators,” she says. It also led to a outflow of skills from the government to systems integrators.

What made all this worse was a view of IT that wasn’t joined up. In practice, Whitehall just hasn’t known which systems it has and how they interoperate with each other, warns Freeguard.

There was also a loss of political will. UK government tech reform looked promising in the early 2010s, following Martha Lane Fox’s report, Directgov 2010 and beyond: revolution not evolution. That recommended a central team to oversee digital transformation across government, including a CEO for digital government operations.

The following year, Whitehall created the Government Digital Service (GDS) to help execute some of those ideas.

“I think there was a lot of momentum around some of this stuff in the early days of GDS,” says Freeguard. There was a lot of political muscle behind it. Then around 2015, things began to fall off.

“The minister was no longer as interested in spending political capital on it, GDS lost some of its authority in terms of spend controls, and there were a few more changes of leadership in GDS as well,” he says. “So, I think some of that initial momentum has been lost, and maybe that has contributed to where we are now.”

Legacy systems, modern challenges

Today, government is caught in a legacy funding trap, warns Chamberlin. That’s caused in part by the governmental belt-tightening we’ve seen of late.

“For many, there is effectively zero money available to tackle legacy technology,” he says. Keeping legacy systems operational costs three or four times as much as maintaining modern systems. Yet the government is spending 30 percent less than its peers on IT overall, per the State of Government analysis.

Another problem is visibility. As the State of Digital Government report pointed out, most organizations don’t know how much technical debt they have in the form of legacy systems. Legacy registries are often non-existent or incomplete, meaning that 15 percent of government survey respondents couldn’t even guess at the size of their legacy burden. That precludes a risk-based approach for modernization.

Current reform efforts

The government restructured its efforts to manage its IT in the past few years. In 2021, it carved out part of the GDS to become the Central Digital and Data Office (CDDO), a separate operation within DSIT to oversee its digital services. It released a 2022-2025 road map for digital services reform.

It had six missions. The first was a set of transformed public services that achieved the right outcomes. As of January, fewer than one in three of the 75 top government services had met conditions that meant they were operating at a “great” standard. The original CDDO goal was two-thirds.

The second mission was the GOV.UK One Login system, which would make it easier to access government systems in a single place. In January it said that 50 services were using One Login, with six million people using it. That’s a big lift from the 3.3 million people that were using it in a February 2024 CDDO progress report [PDF].

Mission number three was the provision of better data for decision making, efficient, secure, and sustainable tech, more digital skills inside government, and a system that unlocks digital transformation.

The government created a centralized Data Marketplace for data, but it’s still in private beta.

In any case, what kind of data are we talking about? “In general, anything that runs on a legacy system is likely not to generate the sort of data you need to do data science,” Margetts says.

There was still no framework for identifying data quality issues as of February 2024. Work was underway on one, according to the CDDO update.

Mission four advocated for efficient, secure, and sustainable technology. This spoke directly to the legacy problem, including a core focus on moving away from legacy systems.

Some work was done here. The government published a legacy system assessment framework in September 2023, with 27 departments on board and providing self-assessments. The CDDO had identified 63 “red rated” systems, and a little over two-thirds had fully funded remediation plans as of 2024.

However the Stage of Digital Government study this January (almost a full year after the CDDO update) was brutally honest. It found that 22 percent of legacy systems were red-rated, which actually rose 16 percent thanks to issues such as known security vulnerabilities and a lack of support.

UK government told to get a grip on £23B tech spend

READ MORE

One way that organizations often tackle the legacy problem is through cloud migration. The CDDO pointed to the government’s Cloud First strategy, published in 2013, citing an average 25-30 percent cloud adoption rate.

Again, all is not what it seems. Most migrations don’t include full refactoring, found the State of Government report. In January this year the National Audit Office also published a report on government procurement and found the government’s approach to the topic troubling.

“Digital services are rapidly changing in nature and are increasingly underpinned by technology and services which are subscription-based and which government does not ultimately control,” it said. “This calls for an approach which responds to this changing environment.”

The Register also found last year that mega-deals with large cloud service providers had resulted in vendor lock-in that hindered its ability to negotiate billions in further cloud spending.

In the meantime, smaller UK-based cloud vendors get left out. They were supposed to get a piece of the pie as part of the its GCloud initiative. At least one key UK operator went out of business after a large government customer moved its business to a US supplier. Others have seen a downturn in their business.

The CDDO also promised to make its use of IT more sustainable (as in, environmentally friendly). However, this seems fundamentally at odds with AI, especially generative AI that chews through electricity and (in datacenters with evaporative cooling), water.

The recent AI Strategy focused more on AI’s potential to find energy savings than it did on reducing energy consumption for AI datacenters. The government ensured it would study the latter, as datacenter operators lined up to build.

“I think there are people in government who are thinking about this, but I’m not sure it’s at the top of the list at the moment,” says Freeguard.

Mission number five: talent. CDDO recognized that the government needed more digital skills. But here, too, it was struggling. It aimed to upskill 90 percent of senior civil servants in digital and data essentials. By February last year it had reached just 9.5 percent. The current plan? A hand-wavy partnership with Google that Whitehall says will help upskill up to 100,000 government employees.

The final mission, a “system that unlocks digital transformation” goal, aimed to embed digital approaches and cross-functional teams into policy design and delivery. When the CDDO update dropped, just over half of the 75 top government services reported having a single owner in place spanning digital, operational, and policy for government services.

Time for a refresh

Now, the CDDO is no more. In January, just four years after its creation, it was reabsorbed by the GDS along with the Geospatial Commission and the Incubator for Artificial Intelligence (i.AI). It still sits under DSIT, but a reboot is afoot. The government wants to re-address its problem.

Its primary vehicle for this is the Blueprint for Modern Digital Government. It’s a six-point plan that reinvigorates GDS as the digital center of government:

1. Join up public sector services: This includes a digital wallet to hold government credentials, and mandated reuse of personal data across all government services (with appropriate safeguards).
2. Harness the power of AI for the public good: This includes building it into public services.
3. Strengthen and extend digital and data public infrastructure: There’ll be a National Data Library and a continued focus on expanding One Login.
4. Elevate leadership, invest in talent: This boils down to “more skills please.”
5. Funding reforms: “We’re piloting four new approaches to fund innovation, including ‘staged funding; for AI and digital projects, building on the success of the experimental GOV.UK Chat tool, to get prototypes off the ground more quickly and avoid the costly delays of outdated approval processes,” DSIT told El Reg.

   It will also make legacy modernization a budget line item.

   “Departmental Spending Review settlements include dedicated funding to replace legacy technology and reduce risk, reflecting the Government’s commitment to strong digital and technology foundations,” the spokesperson continued. “This includes funding for HMRC’s Transformation Roadmap to modernize tax and customs systems.”

   The big switch here will be from “boom and bust” mega-project funding to smaller, iterative funding rounds that reward tangible outcomes. There will also be tailored funding models for modernizing legacy systems.

6. Transparency and accountability: This means doing more government work in the open and being accountable for results, with more regular roadmap publications and updates. There will also be more openness about AI usage.

The government still has a long way to go. In January, satisfaction with 70 percent of the top 75 central government services lagged private sector benchmarks. Almost half of all central government services have no pathway to digital modernization, while in 2024 one in four respondents to the government’s survey had suffered a critical outage.

The government promises it will do better this time. What alternative do we have than to trust it? ®

Original Source