The Big Picture
Combining a code-aware knowledge base with large language model test generation and quantum-aware mutations produces many more valid tests and finds real faults — KQFuzz uncovered 13 bugs (12 fixed) and boosted coverage by up to 18.44%.
ON THIS PAGE
The Evidence
A knowledge-guided approach that feeds library-specific API facts into a language model produces far more valid quantum test programs than naive generation. Adding quantum-aware mutations (parameter corner cases and entanglement-aware gate swaps) plus a fitness function focused on gate and API diversity steers testing toward complex, bug-prone code. Evaluated on Qiskit, PennyLane, and Cirq, the method found 13 real bugs Mutual Verification Pattern (12 have been fixed) and increased coverage compared to prior fuzzers.
Data Highlights
113 real bugs discovered across Qiskit, PennyLane, and Cirq; 12 of them confirmed and fixed by developers.
2Up to 18.44% higher coverage than the prior state-of-the-art fuzzing baselines.
3Out-of-date language-model seed generation yielded only 35–46% valid quantum programs (vs. 64–86% for classical libraries), with ~118 import/attribute errors per 600 generated programs.
What This Means
Engineers and QA teams building or integrating quantum libraries should care because this approach finds real, fixable faults that standard tests miss, through Human-in-the-Loop capabilities. Tool builders and researchers working on automated testing or test generation can reuse the knowledge-guided seed strategy and mutation operators to improve test validity and reach.
Not sure where to start?Get personalized recommendations
Key Figures
Fig 1: Figure 1 . Comparison of Fuzzing Scopes.
Ready to evaluate your AI agents?
Learn how ReputAgent helps teams build trustworthy AI through systematic evaluation.
Learn MoreYes, But...
Results come from experiments on three major Python quantum libraries; behavior may differ for niche frameworks or non-Python ecosystems. KQFuzz depends on the quality and freshness of the language models used — older models produce many invalid seeds and higher cost models were used once to build API summaries. The fitness heuristics focus on structural diversity and entanglement patterns, so purely semantic or hardware-specific bugs may still be missed. The evaluation strategy aligns with Evaluation-Driven Development (EDDOps).
Methodology & More
KQFuzz builds a compact, library-specific knowledge base of API metadata, associations, natural-language summaries, and evolution metrics, then uses that knowledge to guide a language model to generate seed quantum programs that align with the current API surface. It applies a probabilistic API selection that favors semantically associated and rapidly evolving APIs — a likely place for bugs — and ranks seeds with a fitness function that rewards gate diversity, number of entangled qubits, API diversity, and call depth. After seed generation, KQFuzz mutates programs at two levels: parameter-level mutations to hit numeric corner cases and gate-level, entanglement-aware substitutions to explore structural variations. The approach reflects the Blackboard Pattern in organizing and sharing API insights across the system.
Avoid common pitfallsLearn what failures to watch for
Credibility Assessment:
All authors affiliated with HKUST (a recognized university) and includes an author with h-index ~21, indicating an established researcher despite being an arXiv preprint.