Coni: Detecting Database Connector Bugs via State-Aware Test Case Generation
Read the paper · doi:10.1109/icse55347.2025.00003
What this paper does with SQLancer
SQLancer is both the motivating example and a baseline the authors had to build. The paper notes that SQLancer relies on JDBC solely to execute queries and retrieve results, without touching configuration properties or batch execution, which is the gap Coni targets. To measure against it the authors implemented SQLANCER+, adapting the connector and collecting results across connectors to find inconsistencies. It covered 5950 fewer branches than Coni and found none of the 21 bugs -- and the paper explains precisely why, giving a bug that requires calling setFetchSize, a method SQLANCER+ never calls.
Coni tests database connectors -- the JDBC drivers between an application and a DBMS -- rather than the database itself. Its observation is that SQL fuzzers exercise the connector only as a pipe: they open a connection, create a statement and execute a query, leaving batch execution, configuration properties and the rest of the interface untested. Coni generates state-aware sequences of connector API calls and found 21 bugs across five connectors, where the baselines found none.
How it was classified
uses infrastructure — no
M4 describes building SQLANCER+ by adapting SQLancer so that it could serve as a comparison target. Coni's own test generation is its state-aware API sequence construction, not anything built on SQLancer.
extends technique — no
The adaptation exists to make a baseline runnable on connectors, not to develop a SQLancer oracle. Coni's oracle is differential comparison across connectors.
compares with — yes
M4 records SQLANCER+ being implemented as a comparison target, M5 reports the branch coverage difference, and M10 gives the per-connector bug table where it found none.
We implemented SQLANCER+ by adapting the target database connector and collecting results from different connectors to identify inconsistencies.
The result shows that CONI covered 5950, 6608, and 6587 more branches than SQLANCER+, RANDOOP, and EVOSUITE respectively.
Connector CONI SQLANCER+RANDOOP EVOSUITE MariaDB Connector/J 5 0 0 0 MySQL Connector/J 6 0 0 0 AWS MySQL JDBC 3 0 0 0 PGJDBC 2 0 0 0 PGJDBC NG 5 0 0 0 T otal 21 0 0 0 Impro vement-21↑ 21↑ 21↑ In summary, CONI is unique in its ability to find bugs in database connectors, and compared to other techniques, it can cover more branches within the database connectors, which adequately answers RQ2.
describes as state of the art — no
SQLancer is called a popular open-source tool for testing databases using JDBC; popularity is not a state-of-the-art claim.
SQLancer publications it cites (1)
Bibliography entries that resolved to a SQLancer publication, or to a paper by one of the project's authors. A sentence citing one of these numbers is a reference to SQLancer even when it never writes the name.
| # | Entry | Matched as |
|---|---|---|
| 11 | M. Rigger and Z. Su, “Testing Database Engines via Pivoted Query Synthesis,” in 14th USENIX Symposium on Operating Systems Design and Implementation OSDI 20), 2020, pp. 667–682. | sqlancer publication · PQS |
Every place it refers to SQLancer (12)
12 sentences, each stored verbatim from the extracted text with where it was found and how. “Citation marker” means the sentence names no tool at all and was reached through a reference number that resolved to a SQLancer publication.
| Id | Sentence | Found by | Where |
|---|---|---|---|
| M1 | On the one hand, most fuzzers primarily concentrate on generating effective SQL queries [10, 11, 12, 13, 14], whereas database connectors are not directly involved in the execution of these queries. |
citation marker |
I INTRODUCTION page 2 |
| M2 | For example, SQLANCER [11] relies on JDBC solely to execute SQL queries and retrieve results, without exploring additional functionalities like modifying configuration properties or batch execution. |
name |
I INTRODUCTION page 2 |
| M3 | In addition, SQLANCER [11] is a popular open-source tool for testing databases using JDBC. |
name |
C Comparison with Existing Techniques page 8 |
| M4 | We implemented SQLANCER+ by adapting the target database connector and collecting results from different connectors to identify inconsistencies. |
name |
C Comparison with Existing Techniques page 8 |
| M5 | The result shows that CONI covered 5950, 6608, and 6587 more branches than SQLANCER+, RANDOOP, and EVOSUITE respectively. |
name |
C Comparison with Existing Techniques page 8 |
| M6 | Specifically, SQLANCER+ only uses the database connector to send SQL queries, utilizing getConnection to create a database connection, createStatement to create a statement, and execute to execute SQL queries. |
name |
C Comparison with Existing Techniques page 8 |
| M7 | TABLE III: Number of branches covered by each technique in 24 hours Connector CONI SQLANCER+RANDOOP EVOSUITE MariaDB Connector/J 1073 466 583 581 MySQL Connector/J 2430 1256 1473 1489 AWS MySQL JDBC 2826 1445 1734 1739 PGJDBC 1660 987 1197 1181 PGJDBC NG 2425 1796 1621 1597 T otal 10414 5950 6608 6587 Impro vement-4... |
name |
C Comparison with Existing Techniques page 8 |
| M8 | The main reason is as follows: SQLANCER+ did not call the interface methods that can trigger bugs, nor did it attempt to pass illegal values or set configurations for interface methods. |
name |
C Comparison with Existing Techniques page 8 |
| M9 | For example, SQLANCER+ cannot find the 674 bug in Listing 1 because it did not call the setFetchSize method. |
name |
C Comparison with Existing Techniques page 8 |
| M10 | Connector CONI SQLANCER+RANDOOP EVOSUITE MariaDB Connector/J 5 0 0 0 MySQL Connector/J 6 0 0 0 AWS MySQL JDBC 3 0 0 0 PGJDBC 2 0 0 0 PGJDBC NG 5 0 0 0 T otal 21 0 0 0 Impro vement-21↑ 21↑ 21↑ In summary, CONI is unique in its ability to find bugs in database connectors, and compared to other techniques, it can cover... |
name |
C Comparison with Existing Techniques page 9 |
| M11 | When applying fuzzing techniques to test DBMSs, the main challenge is to generate correct and effective SQL queries [11, 12, 13, 35, 36, 37]. |
citation marker |
VII RELATED WORK page 10 |
| M12 | For example, SQLANCER [11] generates valid SQL queries based on AST. |
name |
VII RELATED WORK page 10 |