Artificial intelligence is advancing by leaps and bounds, but with each new leap, uncomfortable questions about data security arise. Especially when we talk about AI agents, those autonomous systems that promise to transform business processes but also expose sensitive information. Until now, the solution seemed simple: encrypt data at rest and in transit. However, the true Achilles' heel has always been processing. What happens when an AI model needs to work with data in memory? That's where confidential computing comes in, a technology that Nvidia has made its flagship.

The Problem of Data in Use

Dion Harris, senior director of high-performance computing at Nvidia, explains it clearly: "70% of data exists outside the cloud, in on-premise data centers and data lakes". To apply AI to this data while maintaining security, confidential computing becomes indispensable. Traditional encryption protects data when stored (at rest) and when traveling over the network (in transit), but during processing, data must be decrypted in memory, exposing it to system administrators and cloud operators. Confidential computing creates a trusted execution environment (TEE) anchored in hardware, where data is only decrypted when strictly necessary for computation and re-encrypted immediately afterward.

This approach is not new, but its application to AI has gained relevance with the arrival of intelligent agents. As Harris points out, "we have gone from generative AI to agentic AI deployed to solve real business problems". In this context, confidential computing acts as an enabler, protecting both data and workload design.

The Performance Bottleneck

Until recently, the main obstacle to adopting confidential computing was the performance impact. Early implementations reduced throughput by 30% to 40%, making it economically unviable for production use. "If you lose that much performance, you reduce the ability to fully utilize the hardware you deploy to generate tokens or provide services efficiently", Harris explains.

But with the arrival of the Blackwell architecture, Nvidia has eliminated this burden. Confidential computing has become a fundamental feature of the system, integrated at the hardware level with no performance penalty. "You get privacy and performance: a double benefit", Harris states. This is key for companies that need to scale their AI models without compromising security, such as those in the financial, healthcare, or government sectors.

A Real Example: Apple and Its Private Cloud Compute

To understand how it works in practice, Harris gives a hypothetical but revealing example: imagine a user wants to upload a medical report to an AI assistant. The system creates a secure verified environment and sends a request to the server. Remote attestation allows the edge device to verify that the GPU is from Nvidia and that the environment has not been tampered with. Once trust is established, data travels encrypted to the GPU memory, where specific compute engines decrypt it only for processing. The LLM model analyzes the report, summarizes it, and the results are re-encrypted before being sent back.

This model, similar to what Apple uses with its Private Cloud Compute and now extends to Google Cloud, combines the power of AI in data centers with the privacy of on-device processing. "It's the best of both worlds", Harris summarizes.

The Future of Confidential Computing in AI

Adoption is growing rapidly, especially in hybrid environments where companies need to combine on-premise resources with public cloud. "We are no longer in fully self-controlled infrastructures", Harris warns. By 2030, confidential computing is expected to generate billions in use cases, becoming essential infrastructure for AI adoption across the industry.

For organizations handling sensitive data or subject to regulations, Harris's recommendation is clear: "Start with a proof of concept. Validate that it works. Plan the deployment". The technology is already mature, with partners like Red Hat and Fortanix integrating it into their platforms, and cloud providers like Google Cloud offering it as a service.

At ForgeNEX, we have analyzed how confidential computing aligns with trends like 12-million-token contexts or SAS's pragmatism in analytics. We have also seen how some projects ban AI agents for fear of losing control. Confidential computing offers a middle path: leveraging AI without sacrificing security.

Conclusion: A Competitive Advantage

Confidential computing is no longer a futuristic option but a strategic necessity for any company wanting to deploy AI on sensitive data. With the performance problem solved, the technology is ready for mass adoption. As Harris concludes, "organizations will gain a competitive advantage by securing AI on sensitive data". In a world where digital trust is currency, confidential computing is shaping up to be the next industry standard.

Original source: ComputerWorld. Analysis and adaptation by ForgeNEX.