WingFuzz: Implementing Continuous Fuzzing for DBMSs
What this paper does with SQLancer
SQLancer is one of three state-of-the-art fuzzers WingFuzz is measured against, with branch counts and bug counts reported per DBMS -- 211,620 more branches and 25 more unique bugs. The paper attributes SQLancer's lower coverage to its generating queries from custom pattern rules tied to the grammar its oracles need, and describes its three oracles, noting NoREC requires a query with WHERE and JOIN clauses.
WingFuzz implements continuous fuzzing for DBMSs, addressing the problem that a fuzzer must keep working as the system under test changes. It combines mutation with feedback guidance and a memory-sanitisation layer, and reports finding 27 unique bugs across PostgreSQL, MySQL, MariaDB and PolarDB during continuous operation.
How it was classified
uses infrastructure — no
SQLancer is cited, not used; nothing in the mentions describes reusing its code.
extends technique — no
No technique is extended; the citation is background.
compares with — yes
M2 names SQLancer among the three fuzzers compared, and M3 and M7 give the results: 211,620 more branches and 25 more unique bugs than SQLancer.
1 Compared with Existing Fuzzers To show the effectiveness of WINGFUZZ, we compared it with three state-of-the-art fuzzers, including conventional mutation-based fuzzer SQUIRREL as well as generated-based fuzzer SQLancer and SQLsmith, which are widely used in the industry to test DBMSs.
They show that WINGFUZZ covers a total of 211620, 197059, 132397 more branches than SQLancer, SQLsmith, and SQUIRREL, respectively.
Specifically, WINGFUZZ finds a total of 25, 24, 21 more unique bugs than SQLancer, SQLsmith, and SQUIRREL, respectively.
describes as state of the art — yes
M2 calls the three compared fuzzers, SQLancer among them, state-of-the-art.
1 Compared with Existing Fuzzers To show the effectiveness of WINGFUZZ, we compared it with three state-of-the-art fuzzers, including conventional mutation-based fuzzer SQUIRREL as well as generated-based fuzzer SQLancer and SQLsmith, which are widely used in the industry to test DBMSs.
SQLancer publications it cites (4)
Bibliography entries that resolved to a SQLancer publication, or to a paper by one of the project's authors. A sentence citing one of these numbers is a reference to SQLancer even when it never writes the name.
| # | Entry | Matched as |
|---|---|---|
| 1 | Bugs found in database management systems. https:// www .manuelrigger .at/dbms-bugs. Accessed: June 7, 2024. | project authored |
| 40 | RIGGER, M., ANDSU, Z.Detecting optimization bugs in database engines via non-optimizing reference engine construction. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Sympo... | sqlancer publication · NOREC |
| 41 | RIGGER, M., ANDSU, Z.Finding bugs in database systems via query partitioning. Proceedings of the ACM on Programming Languages 4, OOPSLA (2020), 1–30. | sqlancer publication · TLP |
| 42 | RIGGER, M., ANDSU, Z.Testing database engines via pivoted query synthesis. In 14th USENIX Symposium on Operating Systems Design and Implementation OSDI 20) (2020), pp. 667–682. | sqlancer publication · PQS |
Every place it refers to SQLancer (11)
11 sentences, each stored verbatim from the extracted text with where it was found and how. “Citation marker” means the sentence names no tool at all and was reached through a reference number that resolved to a SQLancer publication.
| Id | Sentence | Found by | Where |
|---|---|---|---|
| M1 | Consequently, the resilience and security of DBMSs have become paramount considerations for organizations and corporations [1, 5, 9]. |
citation marker project authored |
1 Introduction page 2 |
| M2 | Both the industry and academia have developed many methods for detecting issues in DBMSs [24, 32, 42, 46]. |
citation marker |
1 Introduction page 2 |
| M3 | 1 Compared with Existing Fuzzers To show the effectiveness of WINGFUZZ, we compared it with three state-of-the-art fuzzers, including conventional mutation-based fuzzer SQUIRREL as well as generated-based fuzzer SQLancer and SQLsmith, which are widely used in the industry to test DBMSs. |
name |
6.1 Compared with Existing Fuzzers page 8 |
| M4 | They show that WINGFUZZ covers a total of 211620, 197059, 132397 more branches than SQLancer, SQLsmith, and SQUIRREL, respectively. |
name |
6.1 Compared with Existing Fuzzers page 8 |
| M5 | DBMS SQLancer SQLsmith SQUIRREL WINGFUZZ PostgreSQL 61,420 66,821 63,742 84,954 MySQL 69,182 67,291 77,829 104,281 MariaDB 60,031 69,281 91,932 129,832 PolarDB 56,928 58,729 93,281 140,114 Total 247,561 262,122 326,784 459,181 Increment 211,620 197,059 132,397 – The main reason contributing to the enhanced coverage ... |
name |
6.1 Compared with Existing Fuzzers page 8 |
| M6 | Specifically, SQLancer generates queries based on custom pattern rules, limiting its support to the SQL grammar associated with the defined test oracle. |
name |
6.1 Compared with Existing Fuzzers page 8 |
| M7 | Benefiting from feedback guidance and the data dependency graph, SQUIRREL outperforms both SQLancer and SQLsmith in these DBMSs. |
name |
6.1 Compared with Existing Fuzzers page 8 |
| M8 | Specifically, WINGFUZZ finds a total of 25, 24, 21 more unique bugs than SQLancer, SQLsmith, and SQUIRREL, respectively. |
name |
6.1 Compared with Existing Fuzzers page 9 |
| M9 | DBMS SQLancer SQLsmith SQUIRREL WINGFUZZ PostgreSQL 0 1 1 2 MySQL 1 1 1 7 MariaDB 0 0 2 7 PolarDB 1 1 2 11 Total 2 3 6 27 Increment 25 24 21 – The enhancement in bug triggering primarily stems from two factors. |
name |
6.1 Compared with Existing Fuzzers page 9 |
| M10 | SQLancer [40 –42] designs three test oracles to detect logic errors and generates queries following the oracles. |
name |
8.2 Generation Based DBMS Fuzzing page 13 |
| M11 | For example, its NOREC [40] oracle requires constructing a SQL query with WHERE and JOIN clauses. | technique | 8.2 Generation Based DBMS Fuzzing page 13 |