CutGEN – Intelligent Unit Test Generator

Why Generating Unit Test Cases is Important

In 1996, the Ariane 5 rocket exploded just 37 seconds after launch. The cause? A single line of unhandled code that wasn’t properly tested. Cost: $440 million and years of work, gone in an instant.

Your COBOL programs might not launch rockets, but they do something just as critical, process payrolls, handle millions of financial transactions, and manage inventory worth billions. A single untested code change can cascade into a catastrophe.

Testing isn’t a luxury. It’s a safety net. But for decades-old COBOL applications, creating and maintaining test cases has always been one of the hardest parts of modernization.

COBOL Check

In our previous post, we discussed how COBOL Check modernizes COBOL unit testing by introducing a DSL-based framework that brings test-driven development (TDD) principles to the mainframe. After experimenting with it, we were impressed by its flexibility and the way it integrates seamlessly into modern CI/CD workflows.

However, one big challenge remained – you still have to write the test cases.

Even though Cobol Check’s syntax is straightforward, becoming familiar with its DSL and manually writing .cut files still requires time and effort. Why not leverage AI to automatically generate .cut files that can be fed directly into COBOL Check?

Our Two-Step Approach to Intelligent Test Generation

We developed a two-step process that combines large language models (LLMs), automated validation, and COBOL Check execution to create a closed-loop testing workflow:

  1. Intelligent Test Generation and Validation
  2. Test Execution with Cobol Check

Step 1: Intelligent Test Generation and Validation

We start by providing the large language model with the source COBOL program and, if available, supporting documentation. The model then generates COBOL Check compatible unit test files.

Command to Generate Test Cases:

 

COBOL programs can easily span thousands of lines across many sections. Often, when refactoring a single section, we only need to test that part. To make this scalable, we split the source program by sections (default) or by divisions, allowing the model to focus on generating tests for one logical unit at a time.

Our Source Program:

We don’t use the generated tests right away, they first go through our ANTLR-based COBOL Check test-suite parser, which checks for:

  • Syntax errors – makes sure the test follows Cobol Check format.
  • Variable references – confirms all variables in the test exist in the source program.
  • Duplicates – removes repeated or redundant test cases.


If errors are found, the parser flags them, and the model regenerates the invalid tests. This process repeats through iterative refinement until all tests are valid.

Validated Test Cases

Step 2: Test Execution with Cobol Check

Once the generated .cut files pass validation, we feed them with COBOL Check using the configured properties.

Cobol Check executes the tests, compares actual and expected outputs, and produces detailed reports.

Running COBOL Check:

Conclusion

This setup makes COBOL testing faster, safer, and easier to maintain. By combining AI-generated test cases, automated validation, and COBOL Check execution, we’ve built a practical way to modernize COBOL testing without changing the codebase itself.

This approach helps teams save time, reduce human error, and bring the reliability of modern testing to long-standing COBOL systems.

Many thanks again also to Adikhan Tuichiyev, who’s master thesis set the base for this tool!

Interested in this solution? Feel free to contact us!

 

Transparency Note
This blog post was drafted with the support of generative AI to help structure and formulate the content. However, the technical background was thoroughly researched by our team beforehand, and we consider the topic highly relevant and worth sharing. The final content has been carefully reviewed and approved by us before publication.

Weitere interessante Blogbeiträge