Gemini generated: The Fairytale Story is Easier Said than Done

Cybersecurity in the AI Era: The Secure-by-Design Dream Meets Reality

This blog was also posted on LinkedIn

At Black Hat, Microsoft executive David Weston presented a vision that, at first glance, sounds both logical and compelling: stop trying to win the patching race against AI and instead focus on Secure-by-Construction principles and memory-safe programming languages.

His argument is straightforward. AI can now discover vulnerabilities and turn them into working exploits far faster than most organizations can patch them. If that’s the new reality, constantly chasing vulnerabilities becomes a losing battle. The answer, according to this view, is to build security into software from the very beginning rather than continuously reacting afterwards.

Weston is far from alone in that thinking. Cybersecurity agencies around the world have been making the same argument for years. Reports such as The Case for Memory Safe Roadmaps (jointly published by CISA, the FBI, NSA and Australia’s Cyber Security Centre) and initiatives like the CISA Secure by Design Pledge, signed by hundreds of software vendors, all point in the same direction. Vendors are increasingly expected to publish transition plans, reduce their reliance on memory-unsafe languages and eliminate entire classes of vulnerabilities at their source.

On paper, it’s hard to argue against any of this. Who wouldn’t want software that is secure by design instead of patched after the fact? Who wouldn’t want to eliminate buffer overflows, memory corruption vulnerabilities and other long-standing security problems before they ever reach production?

That’s where the theory starts running into reality.

Hard Reality Overtakes Theory

Once you look beyond the vision, several practical challenges become impossible to ignore. This is where the idea of a rapid transition to memory-safe software starts colliding with the realities of engineering, budgets, and legacy technology.

1. Rewriting Massive Codebases Is Easier Said Than Done

No software vendor is going to rebuild a complex enterprise platform, critical infrastructure application, or operating system such as Windows from scratch. Rewriting tens of millions of lines of code would cost billions, take years, and inevitably introduce new bugs and regressions along the way. Even worse from a business perspective, customers see very little immediate value in return. The funding required for such a project competes directly with investments in new features, performance improvements, and innovation.

Even Microsoft had to quickly dial back expectations recently after reports surfaced claiming one of their engineers converted a million lines of Windows code to Rust in a single month using AI. The reaction from both the community and internal engineering teams was swift and unambiguous: this was merely an isolated research experiment, not a viable roadmap for the Windows operating system.

The challenge is not converting code. The challenge is converting code while preserving decades of functionality, compatibility, integrations, and undocumented edge cases that customers depend on every day.

2. AI to the Rescue? Not So Fast

The idea that artificial intelligence will easily save us by automatically refactoring C code or generating automated patches is dangerously optimistic. Some studies suggest that as many as half of AI-generated patches are insecure, incomplete, or outright broken. While percentages vary between studies and use cases, the broader conclusion remains consistent: generated code still requires extensive human review before it can be trusted in production environments.

Large Language Models are remarkably good at understanding syntax and generating plausible code. What they still struggle with is context. They do not inherently understand years of architectural decisions, business logic, operational constraints, hidden dependencies, or organizational knowledge embedded within large software projects.

As a result, an AI-generated patch may successfully eliminate a memory vulnerability while unintentionally introducing a logic flaw, race condition, privilege escalation path, or operational issue elsewhere. Fixing one problem while quietly creating another is not the kind of automation security teams can blindly rely on.

3. The Rust Foundation is Creaking Under Pressure – Where Are the Tech Giants?

Advocating for a large-scale transition to memory-safe languages such as Rust often overlooks an important reality: software security ultimately depends on people as much as it depends on technology.

Rust has earned its reputation for helping developers avoid entire classes of memory-related vulnerabilities, and for good reason. Major technology companies including Microsoft, Google, and Amazon have signed the CISA Secure by Design Pledge and adopted Rust for critical new components.

However, the ecosystem behind the language faces many of the same challenges that affect large open-source projects everywhere: maintainer burnout, growing complexity, internal political disputes, increasing demands from both commercial and community stakeholders, and a constant struggle to balance innovation with long-term sustainability.

This raises an uncomfortable question. If the industry considers memory-safe languages essential to the future of cybersecurity, who is responsible for sustaining the projects that make that future possible? Signing industry pledges and publishing strategic roadmaps is relatively straightforward. Ensuring that the underlying ecosystem remains healthy for the next decade is considerably more difficult.

The long-term success of Rust will not depend solely on technical excellence. It will depend on whether the organizations benefiting most from its adoption are willing to make sustained investments in the people maintaining it. That means funding, engineering resources, mentorship, governance support, and the time required to prevent key contributors from burning out. Otherwise, we risk creating a situation where a significant portion of the industry’s security strategy depends on a relatively small group of overextended maintainers.

That’s not a criticism of Rust itself. It’s a reminder that every technology, no matter how promising, ultimately relies on people.

Customers Will Be Living with Patches for Years to Come

What does all of this mean for organizations relying on this software? In the short term, very little will change. Despite all the discussions around Secure by Design, memory-safe languages, and AI-assisted development, most organizations will continue running large amounts of existing software for many years to come.

While ‘inherently secure software’ is a long-term goal, AI is accelerating vulnerability discovery for everyone. Security researchers are increasingly using AI to analyze code, identify weaknesses, and uncover vulnerabilities that might previously have taken weeks or months to find. Software vendors are adopting similar capabilities to identify flaws earlier in their own products and development processes. At the same time, threat actors are leveraging many of those same technologies to analyze software, understand patches, and develop exploits at unprecedented speed. The result is that the entire vulnerability lifecycle is accelerating.

While faster vulnerability discovery is ultimately beneficial for the industry, it creates a new operational challenge for organizations. Vulnerabilities are identified and disclosed more quickly, leaving less time to assess risk, validate fixes, test updates, and deploy patches before attackers begin looking for opportunities to exploit them. Patching cycles that were once measured in months are increasingly being compressed into weeks, and in some cases even days. Security teams, IT operations, and CISOs are under increasing pressure to make rapid risk-based decisions while maintaining service availability and business continuity.

For many organizations, the irony is difficult to ignore. The industry is talking about a future where software becomes inherently secure, yet the immediate reality is that patching and vulnerability management are becoming more important than ever. Before things improve, many organizations are likely to experience the opposite: more updates, shorter remediation windows, and higher operational workloads.

A More Realistic Path Forward

The obvious question is whether software vendors can do anything to reduce this growing patch burden. Realistically, they cannot eliminate it overnight. However, they can begin addressing the underlying causes through a combination of innovation, operational security, and long-term modernization efforts.

The answer is not a single technology, framework, or programming language. If anything, the past decades have shown that cybersecurity rarely has silver bullets. Instead, organizations need to accept that security improvement must happen on multiple fronts simultaneously.

  • Track 1: Innovation (New Features): Everything new being developed must unconditionally adhere to strict Secure & Privacy by Design principles, written from line one in modern development practices and memory-safe programming languages.
  • Track 2: Operational Security (Firefighting): A dedicated team focused entirely on triaging, identifying, validating, testing, and deploying fixes for newly discovered vulnerabilities as quickly and safely as possible.
  • Track 3: Structural Refactoring (Incremental Modernization): A specialized team systematically refactoring legacy codebases module by module. Focusing first on high-risk and externally exposed components where security improvements have the greatest impact.

The Price of Closing the Security Gap

Adopting a three-track approach sounds reasonable on paper. Continue innovating, continue patching, and steadily modernize legacy code. The challenge is that organizations must now do all three simultaneously.

Engineering teams cannot simply stop developing new features while they rewrite legacy applications. Nor can they ignore the growing stream of vulnerabilities that require immediate attention. As a result, engineering organizations must divide their resources across innovation, operational security, and long-term modernization efforts at the same time. That requires additional developers, security specialists, quality assurance resources, testing capacity, training, and governance. In other words, it requires investment.

For years, much of the software industry optimized for new features and market growth. Security improvements were often delivered through patches after release rather than being engineered into products from the start. Regulatory frameworks such as the Cyber Resilience Act (CRA) and NIS2 (supplychain risks and security) are beginning to change those incentives by increasing expectations around secure development practices and vendor accountability.

As vendors invest in all three tracks simultaneously, those costs are likely to find their way into the market through higher subscription fees, increased maintenance costs, or expanded security and compliance offerings. While few customers will welcome higher software costs, the alternative is continuing to absorb the costs of breaches, ransomware incidents, emergency remediation efforts, operational disruption, and regulatory penalties.

The industry is gradually moving from a model of paying for failure to a model of paying for resilience.That transition will not be painless, but it may be the most realistic path toward a more secure software ecosystem.

How is your organization navigating this shift? Are you already feeling the acceleration of patching cycles, and are you prepared to pay more for software that is demonstrably built Secure by Design?

References & Sources

  1. AG Connect: Nieuwe aanpak cybersecurity nodig, nu AI de spelregels verandert
  2. Techzine: Microsoft-engineer: Windows wordt niet herschreven naar Rust met AI
  3. AG Connect: ‘De helft van de AI-patches is onveilig’
  4. Veracode: 2026 GenAI Code Security Report: 100+ Models Tested
  5. Georgetown CSET Cybersecurity Risks of AI-Generated Code
  6. Techzine: Techreuzen adopteren Rust, maar het project erachter piept en kraakt
  7. Cyber.gov.au / CISA: The Case for Memory Safe Roadmaps
  8. CISA: Secure by Design Pledge for Software Manufacturers

Scroll to Top