VIREO: Human-in-the-Loop DBMS Fuzzing with Visualization and LLM Support

Jie Liang, Zhiyong Wu, Jingzhou Fu, Chi Zhang, Runpei Miao, Zhuo Su, Yu Jiang, Shuai Ma. 2026. IEEE International Conference on Data Engineering.

Read the paper · doi:10.1109/icde65706.2026.00180

What this paper does with SQLancer

SQLancer is one of four baselines, named among the state-of-the-art DBMS fuzzers VIREO measures against, and run with its default PQS oracle since the comparison is about crashes rather than logic bugs. Results are reported across five systems in both branch coverage and unique bugs. The paper's diagnosis is specific: SQLancer generates queries from fixed grammar rules tied to its oracles, which bounds how deep its exploration goes, and it does not vary configuration options such as --skip-grant-tables, so a class of bugs stays out of reach. SQLancer's oracles are also cited as the generation-based line of work VIREO departs from. VIREO puts a person in the DBMS fuzzing loop. Its premise is that automated fuzzers plateau: after a few hours the coverage curves of SQLsmith, SQLancer, SQUIRREL and LEGO on MariaDB flatten, leaving much of the system untested. VIREO visualises where coverage has stalled and uses an LLM to turn a tester's guidance into new test cases. In three hours it exceeded the 24-hour branch coverage of all four baselines and found 11 unique bugs to their 2-4. Written by claude-opus-5 from the 20 places this paper refers to SQLancer. The quotations below are the paper's own words, stored verbatim when the text was extracted.

How it was classified

uses infrastructure — no

SQLancer is run as a baseline. VIREO's own contribution is a visualisation and LLM-guided loop, not anything built on the SQLancer codebase.

extends technique — no

No SQLancer oracle is extended; VIREO changes how test cases are chosen, not what makes an outcome wrong.

compares with — yes

M9 names SQLancer among the fuzzers VIREO is compared against, M10 records that it was run with its default PQS oracle, and M13 and M18 report the branch-coverage and unique-bug outcomes.

Pivoted Query Synthesis (PQS)

We tried our best to compare VIREO against state-of-the-art DBMS fuzzers, including SQLsmith [61], SQLancer [57], SQUIRREL [70], and LEGO [34]. M9 · A Evaluation Setup · page 8

Although SQLancer was designed for logic bugs, it can also expose crashes and is evaluated using its default PQS oracle [60]. M10 · A Evaluation Setup · page 8

Table II presents the branch coverage results, showing that VIREO achieved in 3 hours results that surpassed the 24-hour runs of SQLsmith, SQLancer, SQUIRREL, and LEGO by 98%, 59%, 38%, and 15%, respectively. M13 · D Compared to Existing DBMS Testing Techniques · page 10

The results indicate that VIREO detected 9, 9, 8, and 7 more unique bugs than SQLsmith, SQLancer, SQUIRREL, and LEGO, respectively. M18 · D Compared to Existing DBMS Testing Techniques · page 10

describes as state of the art — yes

M9 introduces SQLancer as one of the state-of-the-art DBMS fuzzers the evaluation compares against.

We tried our best to compare VIREO against state-of-the-art DBMS fuzzers, including SQLsmith [61], SQLancer [57], SQUIRREL [70], and LEGO [34]. M9 · A Evaluation Setup · page 8

SQLancer publications it cites (6)

Bibliography entries that resolved to a SQLancer publication, or to a paper by one of the project's authors. A sentence citing one of these numbers is a reference to SQLancer even when it never writes the name.

#	Entry	Matched as
7	Jinsheng Ba and Manuel Rigger. 2024. Keep It Simple: Testing Databases via Differential Query Plans. Proceedings of the ACM on Management of Data 2, 3 (2024), 1–26.	sqlancer publication · DQP
19	Jingzhou Fu, Jie Liang, Zhiyong Wu, Yanyang Zhao, Shanshan Li, and Yu Jiang. 2025. Understanding and detecting sql function bugs: using simple boundary arguments to trigger hundreds of dbms bugs. In Proceedings of the...	project authored
57	Manuel Rigger. [n. d.]. SQLancer Website. https: //github.com/sqlancer/sqlancer. Accessed: March 14, 2026.	sqlancer publication
58	Manuel Rigger and Zhendong Su. 2020. Detecting optimization bugs in database engines via non-optimizing reference engine construction. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conf...	sqlancer publication · NOREC
59	Manuel Rigger and Zhendong Su. 2020. Finding bugs in database systems via query partitioning. Proceedings of the ACM on Programming Languages 4, OOPSLA (2020), 1–30.	sqlancer publication · TLP
60	Manuel Rigger and Zhendong Su. 2020. Testing database engines via pivoted query synthesis. In 14th USENIX Symposium on Operating Systems Design and Implementation OSDI 20). 667–682.	sqlancer publication · PQS

Every place it refers to SQLancer (20)

20 sentences, each stored verbatim from the extracted text with where it was found and how. “Citation marker” means the sentence names no tool at all and was reached through a reference number that resolved to a SQLancer publication.

Id	Sentence	Found by	Where
M1	Early DBMS fuzzers like SQLsmith [61] and SQLancer [57] primarily relied on randomly generated SQL test cases.	name background	I INTRODUCTION page 1
M2	Under conventional DBMS fuzzing, after approximately four hours of testing, the coverage achieved by SQLsmith, SQLancer, SQUIRREL, and LEGO on MariaDB exhibits minimal growth.	name motivation	I INTRODUCTION page 1
M3	In 3 hours, VIREO found 98%, 59%, 38%, and 15% more branches and identified 9, 9, 8, and 7 more unique crash bugs than SQLsmith, SQLancer, SQUIRREL, and LEGO did in 24 hours, respectively.	name result comparison	I INTRODUCTION page 2
M4	Generation-based DBMS fuzzing [19, 57, 61, 65, 66, 67] constructs SQL queries using predefined grammars, templates, or probabilistic models.	citation marker background	B DBMS Fuzzing page 3
M5	SQLancer [57] utilizes multiple test oracles [7, 58, 59, 60] to detect logic bugs, generating queries based on these oracles.	name definition	B DBMS Fuzzing page 3
M6	, SQLsmith, SQUIRREL, SQLancer, and LEGO) covers only a limited portion of PostgreSQL after 72 hours, leaving much functionality untested and potential bugs hidden.	name motivation	III MOTIVATION page 3
M7	, SQLsmith, SQUIRREL, SQLancer, and LEGO) on PostgreSQL still leaves a large portion of the code uncovered.	name motivation	III MOTIVATION page 3
M8	Despite running popular fuzzers like SQLsmith and SQLancer on MySQL, these tools failed to trigger the bug, as they neither explore configuration options like --skip-grant-tables nor generate SQL statements combining preparation and privilege-checking operations.	name motivation	III MOTIVATION page 3
M9	We tried our best to compare VIREO against state-of-the-art DBMS fuzzers, including SQLsmith [61], SQLancer [57], SQUIRREL [70], and LEGO [34].	name state of the art	A Evaluation Setup page 8
M10	Although SQLancer was designed for logic bugs, it can also expose crashes and is evaluated using its default PQS oracle [60].	name baseline	A Evaluation Setup page 8
M11	, SQUIRREL, SQLancer, SQLsmith, and LEGO) under the same testing environment for 24 hours.	name baseline	B Evaluation Procedure page 8
M12	Compared to Existing DBMS Testing Techniques We compared VIREO against SQLsmith, SQLancer, SQUIR REL, and LEGO.	name baseline	D Compared to Existing DBMS Testing Techniques page 9
M13	Table II presents the branch coverage results, showing that VIREO achieved in 3 hours results that surpassed the 24-hour runs of SQLsmith, SQLancer, SQUIRREL, and LEGO by 98%, 59%, 38%, and 15%, respectively.	name result comparison	D Compared to Existing DBMS Testing Techniques page 10
M14	Similarly, Table III presents the functions covered, showing that VIREO achieved in 3 hours results that surpassed the 24-hour runs of SQLsmith, SQLancer, SQUIRREL, and LEGO by 95%, 66%, 32%, and 14%, respectively.	name result comparison	D Compared to Existing DBMS Testing Techniques page 10
M15	TABLE IINUMBER OFBRANCHES COVERED BYDIFFERENT FUZZERS (SQLSMITH, SQLANCER, SQUIRREL ,ANDLEGO IN 24HOURS; VIREO IN 3HOURS ) DBMS SQLsmith SQLancer SQUIRREL LEGOVIREO PostgreSQL 54,745 36,844 39,483 52,193 73,041 MySQL 39,445 41,783 43,944 74,136 81,034 MariaDB 40,394 74,364 84,945 91,083 108,238 Percona 47,293 53,784...	name result comparison	D Compared to Existing DBMS Testing Techniques page 10
M16	SQLancer and SQLsmith rely primarily on predefined patterns to generate queries.	name result comparison	D Compared to Existing DBMS Testing Techniques page 10
M17	SQLancer consistently produces queries based on fixed grammar rules, restricting its exploration depth, while SQLsmith predominantly generates only SELECT statements to maintain semantic correctness, severely constraining its exploration breadth.	name result comparison	D Compared to Existing DBMS Testing Techniques page 10
M18	The results indicate that VIREO detected 9, 9, 8, and 7 more unique bugs than SQLsmith, SQLancer, SQUIRREL, and LEGO, respectively.	name result comparison	D Compared to Existing DBMS Testing Techniques page 10
M19	TABLE IVNUMBER OFBUGS COVERED BYDIFFERENT FUZZERS (SQLSMITH, SQLANCER, SQUIRREL ,ANDLEGO IN 24HOURS; VIREO IN 3HOURS ) DBMS SQLsmith SQLancer SQUIRREL LEGOVIREO PostgreSQL 0 0 1 0 1 MySQL 1 0 0 1 1 MariaDB 0 1 1 2 3 Percona 1 1 0 1 3 GreatSQL 0 0 1 0 3 Total 2 2 3 4 11 Increment 9↑ 9↑ 8↑ 7↑ – Similarly, SQLsmith res...	name result comparison	D Compared to Existing DBMS Testing Techniques page 10
M20	SQLancer is primarily designed to detect logic bugs through predefined rule-based queries.	name definition	D Compared to Existing DBMS Testing Techniques page 10

This page is rendered from _data/papers/paper_doi_10_1109_icde65706_2026_00180.json, extracted from supplied pdf. 15 pages, 71 references parsed.