OpenAI improves COBOL migration: what it means for your SMB

This week OpenAI has put the spotlight on a language that many IT teams only know by reference: COBOL. The company has announced results from two internal evaluations that, according to its own account, show a very notable improvement when it comes to migrating COBOL programs to other languages and building software in languages with less code and documentation available to train models. The headline is striking, but it is worth reading it calmly and with an eye on what really affects an SMB or an IT department with limited resources.

What OpenAI has announced, in short

According to the published information, OpenAI has presented two tests. The first is its own COBOL migration evaluation: the GPT-6.1 Sol model would have reached 86.6% accuracy with low reasoning effort, compared to 5.4% for GPT-5.6 Sol, and with a 39% lower cost. With higher reasoning effort, the new model would reach 97.9%. The second test, called MultiLangBench, measures whether a model can build from scratch a SQLite-compatible database engine in 20 programming languages. There, GPT-6 Astra would have achieved an average pass rate of 87.4%, compared to 26.8% for GPT-5.6 Sol, while GPT-6 Luna, the cheapest option, would remain at 40.3% on average, although with an estimated API cost approximately 78% lower.

The important detail: the COBOL evaluation has not been published. OpenAI has released the MultiLangBench environment and suite on GitHub, but not the COBOL test cases. That means a third party cannot inspect or reproduce those results. The company itself acknowledges that applying these capabilities to production software still requires expert review and testing against business rules and system dependencies.

Why COBOL is still a tough nut to crack

COBOL is not a fashionable language, but it supports a huge part of financial operations and business records. According to IBM data cited in the news, more than 40% of online banking systems, 80% of in-person card transactions and 95% of ATM operations rely on COBOL. The problem for AI is twofold: there is less COBOL code and documentation than Python or JavaScript to learn from, and finding developers with real experience is increasingly difficult. In addition, documentation usually lags behind the systems it describes.

That is why the leap announced by OpenAI, if confirmed, would be relevant. But it is not the first company working in this direction: Anthropic already described how Claude Code could assist in COBOL modernization, and AWS introduced AI agents for mainframe migrations in 2024. The novelty here is the declared level of accuracy, not the concept.

What this means for an SMB

For a Spanish SMB, most of the time COBOL does not appear in its own stack, but it does appear in that of its core banking provider, its accounting firm, its old ERP or the sector application that has been running for twenty years. The promise of migrating COBOL with AI may sound like a magic solution, and it is not. What does change is the cost of exploring a modernization: if a model can translate code fragments and propose equivalents in Java, Python or Rust with greater fidelity, the analysis and validation work is reduced, although it does not disappear.

There are three risks that should be kept in mind:

• Non-auditable results. Without public COBOL test cases, we cannot independently verify the 86.6% or 97.9%. An internal benchmark is a signal, not a guarantee.
• Byte-for-byte parity. The evaluation requires that observable outputs match exactly. In production, that parity depends on business rules, dependencies and edge cases that are rarely documented.
• Hidden debt. Migrating code is not migrating the business. The flows, integrations and associated manual processes are usually the most costly part.

What we recommend at ForgeNEX

Our recommendation for IT teams with legacy systems is to treat AI as an accelerator for analysis, not as a substitute for judgment. In practice, this translates into concrete steps:

1. Inventory before migrating. Identify which COBOL programs are critical, what interfaces they have and what documentation exists. Without that map, any automatic translation is a black box.
2. Start with scoped modules. Test assisted translation on small pieces, with your own test suites that compare real outputs against expected ones.
3. Demand traceability. Every change generated by AI must be reviewable, versionable and reversible. If there is no human review with business knowledge, there is no safe migration.
4. Measure total cost, not just API cost. Savings in tokens or translation hours can be diluted if validation and testing skyrocket.
5. Do not depend on a single provider. Benchmarks change and so does leadership. Maintaining the ability to evaluate alternative models is a form of risk management.

At bottom, the news fits a trend we have already been pointing out: generative AI is getting closer to the systems that support the real business, not only to greenfield projects. That demands more rigor, not less. If your team is considering modernizing a legacy application, it is worth reading the results with interest, but also with the same caution with which we approach other technical risks, such as those we discuss in our analysis on AI agent security or in the review of RCE flaws in KSMBD.

The practical conclusion: AI-assisted COBOL migration is no longer a distant promise and is becoming a plausible line of work, but still without enough public evidence to trust it blindly. For an SMB, the sensible decision is not to wait until everything is resolved nor to jump into the void, but to prepare the ground: document, inventory and build tests. That is what turns a benchmark improvement into a real project.

Source: The New Stack. Analysis and adaptation: ForgeNEX.