BugForge: Constructing and Utilizing DBMS Bug Repository to Enhance DBMS Testing
Read the paper · arXiv:2604.03024
What this paper does with SQLancer
SQLancer is one of two state-of-the-art tools BugForge is compared against, run in its FUZZ mode, with branch coverage and bug counts reported per DBMS -- 441,160 branches to SQLancer's 201,299, and 18 bugs to its 11 over the three jointly supported systems. MonetDB is excluded from the totals because SQLancer does not support it.
BugForge builds standardised DBMS bug repositories from bug reports and turns them into test cases. It collects reports, applies syntax-aware processing and input-adaptive extraction to recover raw proofs of concept, and stores structured metadata alongside them; semantic-guided adaptation then refines these into test cases for fuzzing, regression testing and cross-DBMS discovery. The authors integrated 37,632 reports spanning up to 28 years and found 35 previously unknown bugs.
How it was classified
uses infrastructure — no
SQLancer is cited, not used; nothing in the mentions describes reusing its code.
extends technique — no
No technique is extended; the citation is background.
compares with — yes
M3 records the configuration SQLancer was run in, and M6 to M8 give branch coverage and bug counts for each tool across the jointly supported DBMSs.
For SQLancer, we enable its FUZZ mode for testing.
Overall,BugForge outperforms both SQLancer and SQLsmith in branch coverage across all evaluated DBMSs.
Over the three commonly supported DBMSs,BugForge covers 441,160 branches in total, compared with 201,299 for SQLancer and 177,492 for SQLsmith.
Over the three jointly supported DBMSs except MonetDB,BugForgedetected 18 bugs in total, compared with 11 for SQLancer and 6 for SQLsmith.
describes as state of the art — yes
The comparison section is headed as a comparison of state-of-the-art DBMS testing tools, with SQLancer as one of them.
For SQLancer, we enable its FUZZ mode for testing.
SQLancer publications it cites (4)
Bibliography entries that resolved to a SQLancer publication, or to a paper by one of the project's authors. A sentence citing one of these numbers is a reference to SQLancer even when it never writes the name.
| # | Entry | Matched as |
|---|---|---|
| 4 | M. Rigger, “Bugs found in database management systems,” https: //www.manuelrigger.at/dbms-bugs, accessed: April 6, 2026. | project authored |
| 16 | M. Rigger, “Sqlancer website,” https://github.com/sqlancer/sqlancer, accessed: April 6, 2026. | sqlancer publication |
| 18 | M. Rigger and Z. Su, “Detecting optimization bugs in database engines via non-optimizing reference engine construction,” inProceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symp... | sqlancer publication · NOREC |
| 39 | S. Zhong and M. Rigger, “Understanding and reusing test suites across database systems,” 2, no. 6, Dec. 2024. [Online]. Available: https://doi.org/10.1145/3698829 | project authored |
Every place it refers to SQLancer (10)
10 sentences, each stored verbatim from the extracted text with where it was found and how. “Citation marker” means the sentence names no tool at all and was reached through a reference number that resolved to a SQLancer publication.
| Id | Sentence | Found by | Where |
|---|---|---|---|
| M1 | Test researchers, on the other hand, often disclose bugs in public communities and further organize them into structured datasets for analysis, testing, and methodology validation [4]–[6]. |
citation marker project authored |
I INTRODUCTION page 1 |
| M2 | In addition, these DBMSs have long served as standard subjects in prior DBMS testing research [16]–[20]. |
citation marker |
A Implementation and Evaluation Setup page 8 |
| M3 | For SQLancer, we enable its FUZZ mode for testing. |
name |
D Comparison of State-of-the-art DBMS Testing Tools. page 10 |
| M4 | Since SQLancer does not support MonetDB, the total is computed only over the three supported DBMSs, i. |
name |
D Comparison of State-of-the-art DBMS Testing Tools. page 10 |
| M5 | DBMSCovered Branches Detected Bugs SQLancer SQLsmith BugForge SQLancer SQLsmith BugForge MySQL 106,971 96,311 209,720 5 3 8 MariaDB 46,298 30,896 140,677 4 2 7 PostgreSQL 48,030 50,285 90,763 2 1 3 MonetDB N/A 33,209 83,420 N/A 2 5 Total†201,299 177,492 441,160 11 6 18 Table V presents the comparison results. |
name |
D Comparison of State-of-the-art DBMS Testing Tools. page 10 |
| M6 | Overall,BugForge outperforms both SQLancer and SQLsmith in branch coverage across all evaluated DBMSs. |
name |
D Comparison of State-of-the-art DBMS Testing Tools. page 10 |
| M7 | Over the three commonly supported DBMSs,BugForge covers 441,160 branches in total, compared with 201,299 for SQLancer and 177,492 for SQLsmith. |
name |
D Comparison of State-of-the-art DBMS Testing Tools. page 10 |
| M8 | Over the three jointly supported DBMSs except MonetDB,BugForgedetected 18 bugs in total, compared with 11 for SQLancer and 6 for SQLsmith. |
name |
D Comparison of State-of-the-art DBMS Testing Tools. page 10 |
| M9 | Mainstream techniques for database testing include fuzzing [19], [23], [33]–[35], regression testing [17], [36]–[38] and cross-DBMS testing [20], [39], [40]. |
citation marker project authored |
VI RELATEDWORK page 12 |
| M10 | [39] uncover new bugs by integrating and reusing test suites from various open-source DBMSs for cross-testing. |
citation marker project authored |
VI RELATEDWORK page 12 |