OpenAI Reports Stronger Coding Results in COBOL, Fortran, and Visual Basic
OpenAI says its latest models improved on two software-engineering evaluations for low-resource languages. On MultiLangBench, GPT-6 Astra averaged 87.4% at maximum reasoning effort across 20 programming languages; on a COBOL migration test, GPT-6.1 Sol scored 86.6% at low reasoning effort. These are OpenAI-designed evaluations, not a guarantee of production migration quality.
OpenAI's October 19 research update reports stronger results on software tasks in low-resource programming languages, including COBOL, Fortran, and Visual Basic. At maximum reasoning effort, GPT-6 Astra averaged 87.4% on OpenAI's MultiLangBench evaluation across 20 languages, while GPT-6.1 Sol scored 86.6% on a COBOL migration test at low reasoning effort. (OpenAI's evaluation results)
Key Takeaways
- This is an evaluation update, not a new model launch. OpenAI published results for building and migrating software in languages with less training data and documentation than mainstream languages. (OpenAI's research post)
- MultiLangBench tests 20 languages. It measures whether a model can build a SQLite-compatible database engine and pass the adapted test suite. (OpenAI's evaluation description)
- GPT-6 Astra led OpenAI's reported average at maximum reasoning effort. It achieved an 87.4% test pass rate, compared with 26.8% for GPT-5.6 Sol. (OpenAI's benchmark results)
- The COBOL migration result was also higher. GPT-6.1 Sol reached 86.6% accuracy at low reasoning effort, versus 5.4% for GPT-5.6 Sol at the same setting. (OpenAI's migration results)
- Treat the scores as a reason to run your own tests. The evaluations do not establish that a model can safely migrate a particular production system without review.
What OpenAI Measured
OpenAI's MultiLangBench evaluation asks models to build a SQLite-compatible database engine from scratch in 20 programming languages. It measures test pass rate and API cost over an extended coding session, using tests adapted from SQLite's existing test suite. That gives developers a way to compare both task performance and estimated cost across language-specific runs. (OpenAI's MultiLangBench description)
The second evaluation tests whether a model can migrate an existing COBOL program to C++, Java, Kotlin, Python, or Rust while preserving its behavior. OpenAI says hidden tests require byte-for-byte parity on observable outputs, so a migration succeeds only when all tests pass. (OpenAI's COBOL migration description)
Results OpenAI Reports
The results show gains on both tasks, but the headline percentages represent different measures: MultiLangBench reports test pass rate for building a database engine, while the migration evaluation reports accuracy for translating COBOL and preserving outputs. Read each figure within its own test rather than comparing the rows as if they shared one scoring scale. (OpenAI's evaluation results)
| Evaluation | Reported result | What it tests |
|---|---|---|
| MultiLangBench | At maximum reasoning effort, GPT-6 Astra averaged 87.4%, compared with 26.8% for GPT-5.6 Sol, across 20 languages. (OpenAI) | Building a SQLite-compatible database engine and passing adapted tests. |
| MultiLangBench cost | At maximum reasoning effort, GPT-6 Luna averaged 40.3% at an estimated $0.77 per sample, compared with $3.48 per sample for GPT-5.6 Sol. (OpenAI) | A lower-cost model's results on the same evaluation. |
| COBOL migration | GPT-6.1 Sol scored 86.6% at low reasoning effort versus 5.4% for GPT-5.6 Sol; at higher effort, GPT-6.1 Sol reached 97.9%. (OpenAI) | Moving COBOL programs to another language while preserving observable behavior. |
These are OpenAI-reported results from its own evaluations. They are useful for deciding what to test, but they are not an independent comparison or a promise that a migration will preserve every application's business rules.
What Legacy-System Teams Should Do
Use the results to build a small evaluation around your own code and acceptance tests before choosing a model for legacy-system work. OpenAI's migration test focuses on preserving observable behavior, while real systems may also depend on business rules, integrations, and operational constraints that are specific to the codebase. Compare generated output against the original program and review edge cases before deployment. (OpenAI's COBOL migration discussion)
For a migration, compare generated changes against the existing program's outputs and have engineers review edge cases before deployment. OpenAI says teams can try its latest models through the API on coding tasks in the languages they use; the API catalog currently lists GPT-6 Astra, GPT-6.1 Sol, and GPT-6 Luna. (OpenAI API model catalog)
Related AI Catchup coverage: GPT-6.1 Sol and GPT-6 Astra. For this article's language-specific evaluation, see OpenAI's research post.
Sources
- OpenAI, Advancing coding in low-resource programming languages (October 19, 2026)
- OpenAI Developers, Models
Read next
Related coverage- OpenAI
GPT-6.1 Sol: Astra-Level Coding at a Fifth of the Price, With Cost at Every Effort Level
OpenAI released GPT-6.1 Sol on September 29, 2026, at $2 input and $10 output per million tokens, a fifth of GPT-6 Astra's price, with cached input at $0.10. On OpenAI's charts it edges past Astra's best DeepSWE score at high effort and trails Astra by 2.1 points on OSWorld offline. The API model ID is gpt-6.1-sol.
- OpenAI
OpenAI Launches GPT-6 Astra for Computer Use and Complex Work
OpenAI launched GPT-6 Astra on September 3, 2026, for computer use and complex work. Its October 16 update to GPT-6 Pro in ChatGPT adds faster responses, improved plugin and tool use, and parallel agents, rolling out to eligible Pro users.
Frequently Asked Questions
What did OpenAI announce about coding in low-resource languages?
OpenAI published results from two evaluations: building a SQLite-compatible database engine across 20 programming languages and migrating COBOL programs while preserving their behavior. The post reports improvements for GPT-6 Astra, GPT-6 Luna, and GPT-6.1 Sol on those tasks.
How did GPT-6 Astra score on MultiLangBench?
At maximum reasoning effort, OpenAI reports an average test pass rate of 87.4% for GPT-6 Astra across 20 languages, compared with 26.8% for GPT-5.6 Sol. These figures come from OpenAI's own evaluation for building a SQLite-compatible database engine.
How did GPT-6.1 Sol perform on COBOL migration?
OpenAI reports 86.6% accuracy for GPT-6.1 Sol at low reasoning effort, compared with 5.4% for GPT-5.6 Sol at the same setting. At higher reasoning effort, GPT-6.1 Sol reached 97.9% on the evaluation.
Can developers try the models on their own programming languages?
OpenAI says developers can try its latest models through the API on coding tasks in the languages their teams use. Its API model catalog lists GPT-6 Astra, GPT-6.1 Sol, and GPT-6 Luna, but the evaluation results do not guarantee performance on a particular codebase.