Papers calling SQLancer state of the art
Recognition rather than reuse: these papers describe SQLancer or one of its techniques as the state of the art, the most effective, or the approach that established the field. That is a different claim from building on it, so it counts towards no figure on the impact page — not the 30 that reuse or extend SQLancer, nor the 55 that compare against it. Each entry quotes the sentence itself.
Every paper (34)
- BugForge: Constructing and Utilizing DBMS Bug Repository to Enhance DBMS Testing (2026) ·
SQLancer is one of two state-of-the-art tools BugForge is compared against, run in its FUZZ mode, with branch coverage and bug counts reported per DBMS -- 441,160 branches to SQLancer's 201,299, and 18 bugs to its 11 over the three jointly supported systems. MonetDB is excluded from the totals because SQLancer does not support it.
For SQLancer, we enable its FUZZ mode for testing.
— source - Detecting DBMS Bugs by Constructing Equivalent Representations of Intermediate Query Results (2026)
TLP is one of ERIQ's four baselines, named among state-of-the-art DBMS logic-bug detection approaches alongside EDC, Radar and EET, and included partly because EDC's own evaluation used it. The comparison is a careful one: rather than running each tool, the authors reconstruct test cases from each approach's relation and construction procedure starting from the same minimised cases. TLP detected 9 of ERIQ's 54 bugs -- more than any other baseline, and still a sixth of the total. PQS, NoREC and TLP are each described in related work as the query-level oracles ERIQ is positioned against.
We further compared ERIQ with four state-of-the-art DBMS logic bug detection approaches: EDC [ 4], Radar [ 34], EET [ 11], and TLP [ 30].
— sourceWe selected 4 state-of-the-art DBMS logic bug detection approaches as baselines: EDC [ 4], Radar [ 34], EET [ 11], and TLP [ 30].
— source - DIRT: Database-Integrated Random Testing (2026) ·
DIRT takes SQLancer as both its point of departure and its baseline: three of its five oracles are reimplementations of SQLancer's, expressed as generation actions, and it explicitly declines to modify SQLancer itself, arguing that integrating a new DBMS into it is too laborious for a system under active development. It is careful to say that outperforming SQLancer on Turso does not mean it has better generators or oracles.
Within prior work, SQLancer is distinguished by using specialized test oracles that reveal logic bugs in DBMSs, and it has proven extremely successful at finding bugs in mature databases.
— source - Efficiently Detecting DBMS Bugs through Bottom-up Syntax-based SQL Generation (2026) ·
SQLancer is the paper's reference point for the template-based approach it is arguing against, and it is unusually direct about SQLancer's standing: the most advanced tool in the category, and the most popular platform for implementing the latest SQL testing techniques. Its baselines are two of those techniques rather than SQLancer's own oracles -- SQLancer+QPG on CockroachDB and SQLancer+DQP on MySQL and MariaDB -- and the paper notes both are built on top of SQLancer and use its SQL templates. The critique is that those hand-written Java templates bound which SQL features can appear at all, which is the limitation the grammar-based generation is meant to remove.
The most advanced tool in this category is SQLancer.
— sourceAlthough SQLancer was first introduced in 2022, it has become the most popular platform for implementing the latest SQL testing techniques [52]–[56].
— source - Finding Missed Optimizations in DBMSs through Unbalanced Short-circuit Query Construction (2026) ·
CERT is SCor's most direct comparison, one of five performance-bug detection tools it is measured against and described as tackling performance bugs through cardinality estimation analysis. SCor found 36 more bugs than CERT, and the paper gives a precise account of the difference in reach: CERT cannot detect optimization opportunities that persist despite correct cardinality estimates, which is exactly the class short-circuit construction targets. SQLancer also appears as a data-generation source the paper declines to reimplement, listing it among the existing methods available for producing test data.
We compared SCor with five state-of-the-art DBMS performance bug detection tools, including: APOLLO [ 18], AMOEBA [ 24], and CERT [ 1], Puppy [ 54], and HULK [ 55].
— source - FuzzySQL: Uncovering Hidden Vulnerabilities in DBMS Special Features with LLM-Driven Fuzzing (2026) ·
SQLancer is one of three state-of-the-art open-source baselines, alongside Squirrel and EET. The comparison required accommodation in both directions: because SQLancer is not a grey-box tool, the authors collected its generated test cases and replayed them through FuzzySQL's own mechanism to measure coverage, and because SQLancer pairs different oracles with different systems they ran TLP for MySQL and NoREC for the other targets. That constraint is noted as a property of SQLancer's design rather than a shortcoming. Bug counts are reported per target over 24 hours.
To demonstrate the advantage of FuzzySQL, we compare it against three state-of-the-art opensource fuzzing baselines: Squirrel [ 56], EET [ 19], and SQLancer [ 35].
— source - LLMSQLMUTATOR: LLM-Powered Test Case Generation for Database Using Bug Reports (2026) ·
SQLancer is both a baseline and a component. As a baseline it is one of four tools compared over 48 hours, and the paper is even-handed: SQLancer's syntactic correctness exceeds 90% on every system tested, higher than the mutation-based tools, while its coverage peaks around 52%. It is called the most effective black-box fuzzing tool, distinguished by its three complementary oracles. Those oracles are also reused directly: for test cases containing SELECT queries, LLMSQLMutator randomly selects one of TLP, NoREC or CERT as its oracle, describing them as the three most effective existing oracles for detecting result inconsistency. Of its 14 result-inconsistency bugs, TLP found 12 and CERT the other two.
In recent years, SQLancer [11] has emerged as the most effective black-box fuzzing tool, distinguished by its adoption of three complementary oracles [14], [12], [13].
— source - One DBMS, Two Modes, and a Bunch of Bugs: Catching Logic Bugs in Distributed DBMSs via Differential Testing (2026) ·
QPG is DistSQL's closest comparison and its point of departure. The paper describes QPG as improving SQLancer by using query plan text as feedback to mutate the database state when no new plans appear, and argues this focuses on centralized database state and so becomes noisy and less informative on distributed systems. That claim is tested directly: one of DistSQL's own ablation variants uses query-plan-text feedback identical to QPG's, and QPG is also run as an external baseline alongside Jepsen, Mallory and EET, with DistSQL finding 28 more bugs. NoREC, TLP, PQS and CODDTest appear as the local-DBMS oracles the distributed setting is contrasted with.
We compare DistSQL against SOTA database and distributed system testing tools, including Squirrel [ 65], QPG [ 2], and EET [ 24], as well as distributed system testing tools, including Jepsen [20] and Mallory [36].
— source - Operationalizing Property-Based Testing for Data-Intensive Scalable Computing Systems (2026) ·
SQLancer is the demonstration the paper builds on -- credited with showing the effectiveness of query oracles including TLP -- and is explicitly excluded from the evaluation: the authors say they do not compare against it because using it for Spark would require an additional adapter.
In relational DBMS testing, SQLancer [ 32] demonstrated the effectiveness of query oracles, including TLP [ 2], which checks result preservation under predicate-based partitioning, and NoREC [ 3], which validates queries through equivalence-preserving rewrites that disable optimizer-dependent execution paths.
— source - VIREO: Human-in-the-Loop DBMS Fuzzing with Visualization and LLM Support (2026) ·
SQLancer is one of four baselines, named among the state-of-the-art DBMS fuzzers VIREO measures against, and run with its default PQS oracle since the comparison is about crashes rather than logic bugs. Results are reported across five systems in both branch coverage and unique bugs. The paper's diagnosis is specific: SQLancer generates queries from fixed grammar rules tied to its oracles, which bounds how deep its exploration goes, and it does not vary configuration options such as --skip-grant-tables, so a class of bugs stays out of reach. SQLancer's oracles are also cited as the generation-based line of work VIREO departs from.
We tried our best to compare VIREO against state-of-the-art DBMS fuzzers, including SQLsmith [61], SQLancer [57], SQUIRREL [70], and LEGO [34].
— source - Automated Discovery of Test Oracles for Database Management Systems Using LLMs (2025) ·
Argus uses SQLancer++'s query generator to produce seed queries while deliberately not using its predefined oracles, since discovering oracles is the paper's own subject. It measures itself against SQLancer, SQLancer++ and EET on coverage, and calls SQLancer a state-of-the-art open-source DBMS testing framework. Its artifact carries SQLancer's own source files.
As a state-of-the-art, open-source DBMS testing framework, SQLancer supports most of the latest test oracles [ 4,57,74].
— source - DepState: Detecting Synchronization Failure Bugs in Distributed Database Management Systems (2025) ·
SQLancer is one of five baselines and is called a state-of-the-art DBMS testing technique. Because it was built for single-node systems, the authors ran it against one SQL server of each cluster. The comparison is reported in both bugs and coverage of synchronization-related functions, and the paper's explanation for the gap is structural rather than dismissive: SQLancer, SQLsmith and Mozi only generate SQL queries, which cannot navigate the space of cluster states, so all three of SQLancer's bugs were ordinary ones and none was a synchronization failure.
To evaluate the effectiveness of DepState in testing DDBMSs, we compare it with five state-of-the-art tools: Jepsen [ 17],Mallory [26], SQLsmith [ 2], SQLancer [ 33], andMozi [23].
— sourceSQLsmith, SQLancer, and Mozi are three state-of-the-art DBMS testing techniques, which have detected hundreds of bugs in practice.
— source - Detecting DBMS bugs with context-sensitive instantiation and multi-plan execution (2025) ·
Kangaroo argues that SQLancer's oracles constrain the SQL they can test -- NoREC needing a WHERE clause, all three putting limits on queries -- and positions multi-plan execution as free of that constraint. SQLancer is then one of three tools it is measured against over 24 hours, run with NoREC, or TLP where NoREC is unavailable.
Why Existing Works Cannot Detect the Bug NoREC is one of the most effective DBMS logic bug detection oracles which requires the SELECT statements to satisfy some predefined rules, e.
— sourceComparisons with Existing Tools We compare Kangaroo with three state-of-the-art and open source systems: Squirrel, SQLancer, and SQLRight.
— source - Detecting Logic Bugs in DBMSs via Equivalent Data Construction (2025) ·
TLP is one of EDC's three baselines, named as a state-of-the-art open-source tool for finding logic bugs alongside EET and Radar. The comparison is reported in detail and is unusually candid about where each wins: EDC found 38 bugs to TLP's 9, they overlap on 3 -- all SQL-function bugs -- and the 6 bugs only TLP found involve JOINs, which are outside EDC's scope because it tests single-table queries. NoREC, TLP and PQS are each also described in the related-work section as the query-level approaches EDC is positioned against.
We compare EDCagainst 3 state-of-the-art open-source tools for finding logic bugs in DBMSs: TLP [ 37], EET [ 27], and Radar [ 45].
— source - Hulk: Exploring Data-Sensitive Performance Anomalies in DBMSs via Data-Driven Analysis (2025) ·
CERT is the closest prior work and the baseline, and the paper notes it is implemented in SQLancer, referring to it throughout as SQLancer-CERT -- one of only two open-source tools available for the comparison. SQLancer's own oracles are described in related work as the logic-bug counterpart to Hulk's performance focus.
In addition, we compare Hulk with the state-of-the-art DBMS validation tools in industry, including both DBMS performance testing tool APOLLO [ 30] and SQLancer𝐶𝐸𝑅𝑇[11], as well as DBMS fuzzing tools Sqirrel [60].
— source - QTRAN: Extending Metamorphic-Oracle Based Logical Bug Detection Techniques for Multiple-DBMS Dialect Support (2025) ·
SQLancer's oracles are precisely what QTRAN extends: NoREC and TLP are two of the four state-of-the-art techniques it takes and carries to eight DBMSs, and the paper tabulates which systems each currently supports to show the gap it closes. SQLancer is also its measure of the per-DBMS adaptation cost.
Notably, the metamorphic testing based approach MOLT has been recognized to be state-of-the-art in DBMS testing for logical bug detection [ 20,38,40].
— sourceIn our evaluation, we selected four state-of-the-art MOLT s for extension: NoRec [ 37], TLP [38], Pinolo [20], and DQE [44].
— source - Semantic Hint-Based Fuzzing for Time-Series Databases (2025) ·
SQLancer is the pioneer the paper builds from and the limitation it works around: it is credited with pioneering systematic bug detection through oracles such as NoREC and TLP, and then said to assume relational semantics, static schemas and Boolean logic, which is what makes it ineffective for time-series data.
SQLancer pioneered systematic bug detection using oracles such as Non-Optimizing Reference Engine Construction (NoREC) and Ternary Logic Partitioning (TLP), enabling reliable detection of logic bugs even when databases do not crash [6].
— source - Sembug: Detecting Logic Bugs in Dbms Through Generating Semantic-Aware Non-Optimizing Query (2025) ·
SemBug is a direct descendant of NoREC: it says it transforms the optimized query into a less optimized one 'inspired by NoREC', and its evaluation treats NoREC, TLP and Pinolo as the state-of-the-art techniques to beat. It also runs on SQLancer's own machinery, using it as the query generator, and its artifact carries SQLancer's source under a renamed package.
3)Baselines: We compare SemBug against the state-ofthe-art logic bug detection techniques, namely NoREC [18], TLP [10], and Pinolo [19], respectively.
— source - SQLaw: Detecting Bugs in GPU Database Management Systems via Rule-Based Differential Execution (2025) ·
SQLaw imitates SQLancer's statement-level, syntax-based reduction to simplify its own bug reports, and takes NoREC and TLP as two of its three baselines, describing them as representative state-of-the-art approaches that have each found over a hundred bugs in non-GPU DBMSs.
We selected three representative state-ofthe-art approaches, including SQLsmith ,NoREC and TLP .
— sourceI n particular, SQLsmith is one of the most effective approaches for detecting crash and error bugs, while both NoREC and TLP represent advanced approaches for detecting logic bugs.
— source - SRS: Detecting Logic Bugs of Join Implementation in DBMSs via Set Relation Synthesis (2025) ·
SRS is implemented on SQLancer, generating its database state with SQLancer's syntax-rule-based random generation and keeping SQLancer's query generation strategy for features outside the join. SQLancer's oracles are then also its baselines: PQS, NoREC, TLP and DQP are four of the six approaches it is measured against, and it reports finding 9 to 12 more bugs than each.
TQS [ 28] and DQP [ 3] are two state-of-the-art testing approaches that target logic bugs in the optimization of join operations.
— sourceWe selected six state-of-the-art approaches for comparison: PQS [ 25], TLP [ 24], NoREC [ 23], Pinolo [ 11], EET [ 13], and DQP [ 3].
— source - Thanos: DBMS Bug Detection via Storage Engine Rotation Based Differential Testing (2025) ·
SQLancer is one of Thanos's three baselines and is named a state-of-the-art DBMS testing tool throughout. The comparison is reported per system in both bugs and branch coverage, and SQLancer found none of the 14 bugs Thanos did. The paper's argument for why is about where the oracles apply: SQLancer's rely on specially crafted rules -- NoREC on optimizer behaviour, and it describes TLP as needing a pivot row -- which leaves storage-engine differences outside their reach, and its use of standard SQL syntax bounds the code it exercises. SQLancer's construction of functionally equivalent queries is also cited as the metamorphic-testing line Thanos departs from.
We evaluate THANOS on three widely used and extensively tested DBMSs, namely MySQL, MariaDB, and Percona against state-of-the-art fuzzers SQLancer, SQLsmith, and SQUIRREL.
— sourceTo assess the effectiveness of THANOS, we compare THANOS against contemporary state-of-the-art DBMS testing methods, namely SQLancer, SQLsmith, and SQUIRREL.
— source - Understanding and Detecting SQL Function Bugs: Using Simple Boundary Arguments to Trigger Hundreds of DBMS Bugs (2025) ·
SQLancer is one of three baselines, described as a state-of-the-art DBMS testing tool widely used in industry, and run in PQS mode with default settings. The result is stark and reported twice: over 24 hours SQLancer found no SQL function bugs at all, against Soft's 2,956, and Soft covered many times more branches in the function components. The paper's explanation is concrete rather than dismissive -- supporting a new function in SQLancer means writing a function model in Java, and it only generates random values for function arguments, so the boundary values these bugs need are not something it produces.
5 Comparison with Other Testing Works To demonstrate the effectiveness of our methods, we compared Soft against three state-of-the-art DBMS testing tools, namely Sqirrel, SQLancer, and SQLsmith, which are widely used in the industry.
— source - Unveiling Logic Bugs in SPJG Query Optimizations within DBMS (2025) ·
SQLancer is the paper's reference point and its baseline: four of SQLancer's methods -- PQS, TLP, NoREC and DQP -- are the four baselines TQS is measured against, and the paper calls SQLancer the state-of-the-art approach for detecting logic bugs while arguing it is not designed for multi-table queries.
We compare TQS with SQLancer,3which is the state-of-the-art approach to detecting logic bugs in databases.
— source - Semantic Conformance Testing of Relational DBMS (2024) ·
SEMCONT is built on SQLancer, whose syntax-guided generation it enhances with coverage guidance and uses as the seed pool for its own query generation. TLP and NoREC are then its baselines, described as state-of-the-art metamorphic testing methods and run in both their SQLancer and SQLRight implementations.
Coverage guided test case generation The state-of-the-art practice in test case generation involves randomly generating SQL queries guided by the syntax of SQL, among which SQLancer [14] stands out as one of the most effective tools of this kind.
— sourceWe compared SEMCONT with TLP [41] and NoREC [40], which are state-of-the-art metamorphic testing methods for testing RDBMS.
— source - Step-wise Execution of Data-Centric Systems (2024) ·
NoREC and TLP are named as state-of-the-art approaches for detecting logic bugs in DBMSs, the line of work the step-wise reference construction is set against.
NoREC [ 7], TLP [ 8], and TQS [ 9] are all stateof-the-art approaches for detecting logic bugs in DBMSs.
— source - Testing Graph Database Systems via Equivalent Query Rewriting (2024) ·
The paper works from Ternary Logic Partitioning throughout, but at one remove: its comparison target is GDBMeter, which carries TLP to graph databases, and the paper is precise that GDBMeter is based on Query Partitioning and reuses a test oracle designed for relational systems. That reuse is its critique -- it reports that none of the graph-related bugs were detected by GDBMeter for exactly that reason, that TLP applies only to filter clauses, and that partitioning yields four distinct query plans against GRev's many. It states outright that GRev cannot be compared with relational tools such as SQLancer, so SQLancer itself is not a baseline; TLP is, through GDBMeter.
Based on metamorphic testing, GDBMeter [ 15] is the SOTA tool for detecting logic bugs in GDBMS.
— source - Testing Graph Database Systems with Graph-State Persistence Oracle (2024) ·
TLP reaches this paper as the technique GDBMeter carried into graph databases -- the paper describes GDBMeter as the first metamorphic approach to apply TLP from relational DBMSs to graph DBMSs, finding 40 previously unknown bugs, and notes elsewhere that TLP was first proposed for relational systems and has since been applied to graph ones. PQS and NoREC are described alongside as the relational approaches, with PQS called general and highly effective. GraspDB's own baselines are GDsmith and GraphGenie rather than GDBMeter, on the stated grounds that GraphGenie has shown state-of-the-art performance and outperforms GDBMeter.
(PQS) isageneralandhighly-effectiveapproachto/f_indingbugsinDBMS.
— source - WingFuzz: Implementing Continuous Fuzzing for DBMSs (2024) ·
SQLancer is one of three state-of-the-art fuzzers WingFuzz is measured against, with branch counts and bug counts reported per DBMS -- 211,620 more branches and 25 more unique bugs. The paper attributes SQLancer's lower coverage to its generating queries from custom pattern rules tied to the grammar its oracles need, and describes its three oracles, noting NoREC requires a query with WHERE and JOIN clauses.
1 Compared with Existing Fuzzers To show the effectiveness of WINGFUZZ, we compared it with three state-of-the-art fuzzers, including conventional mutation-based fuzzer SQUIRREL as well as generated-based fuzzer SQLancer and SQLsmith, which are widely used in the industry to test DBMSs.
— source - Detecting Logic Bugs of Join Optimizations in DBMS (2023) ·
SQLancer is the work TQS is built against and the paper says so directly, calling it the current state-of-the-art tool for finding logic bugs in DBMSs and the most closely related work to its own. All three of its oracles serve as baselines -- PQS, TLP and NoREC -- and the paper is careful about the comparison's mechanics, noting that SQLancer supports different oracles on different systems and that its generated joins often return empty results and so cannot be used for testing. The gap TQS identifies is single-table scope: SQLancer is not designed to test multi-table queries, which is exactly where join-optimisation bugs live.
SQLancer [ 49] is a current stateof-the-art tool in testing DBMS for logic bugs and is the most closely related work to ours.
— source - Pinolo: Detecting Logical Bugs in Database Management Systems with Approximate Query Synthesis (2023) ·
Pinolo positions itself directly against SQLancer's oracles: PQS is the oracle-guided synthesis it says considers only one row at a time, NoREC and TLP are the metamorphic approaches it says preserve too much of the query, and it states that metamorphic testing of that kind is recognised as the state of the art in DBMS logical bug detection.
Notably, the metamorphic testing based approach has been recognized to be state-of-the-art in DBMS testing for logical bug detection [ 35,37].
— source - Sequence-Oriented DBMS Fuzzing (2023) ·
SQLancer is one of the three fuzzers Lego is measured against, described as a state-of-the-art academic fuzzer that generates from custom rules mainly for SELECT statements. Lego reports covering 198% more branches than it, and notes that SQLancer found no bugs in their setting.
To encompass as many state-of-the-art DBMS fuzzers as possible, we compared LEGO to popular fuzzer SQUIRREL and SQLancer from the academy and SQLsmith from the industry.
— source - Automatic Detection of Performance Bugs in Database Systems using Equivalent Queries (2022) ·
TLP is one of AMOEBA's three baselines and the paper calls it the state-of-the-art tool for finding logic bugs in DBMSs using metamorphic testing. Because TLP also produces equivalent queries, it can be repurposed as a source of pairs, and the authors built a baseline of 2000 TLP-generated pairs to test that. The finding is a clean negative result about transfer: TLP's mutation rules were not designed for performance, and its mutant queries are on average 17 times slower than their base query, so that inherent overhead swamps the signal a performance oracle depends on. AMOEBA found 25 and 14 bugs against the TLP benchmark's 1 and 1.
TLP is the state-of-the-art tool for discovering logic bugs in DBMS using metamorphic testing [ 38].
— source - Detecting Logical Bugs of DBMS with Coverage-based Guidance (2022) ·
SQLRight is built on SQLancer and Squirrel, and two of its four oracles -- NoREC and TLP -- are ported from SQLancer directly. It then measures itself against SQLancer as the state-of-the-art oracle-based tool, reporting more bugs but acknowledging SQLancer's much higher query validity.
We also compare our system with the stateof-the-art tools, including SQLancer (using oracles to detect logical bugs) and Squirrel (using code coverage to detect crashes and assertion failures).
— sourceFor Q2, we compare SQLRight with SQLancer and Squirrel, the state-of-the-art bug-finding tools for DBMSs.
— source - Unicorn: detect runtime errors in time-series databases with hybrid input synthesis (2022) ·
SQLancer is Unicorn's principal baseline. The authors adapted it to each of the six time-series systems and report coverage and bug counts side by side across the whole evaluation, and they call it a state-of-the-art database fuzzer when introducing that comparison. Their explanation for the gap is about scope rather than quality: SQLancer targets the relational model, so its generated queries lack time-series elements and most are invalid on these systems, and it cannot address KairosDB at all because that system takes JSON over HTTP rather than SQL. They also cite SQLancer's three test oracles as the prior work on oracle design.
•We evaluate Unicorn on 6 popular time-series databases against state-of-the-art fuzzers SQLsmith and SQLancer.
— source6 EVALUATION In this section, we evaluate the effectiveness of Unicorn in terms of coverage and bug discovery against state-of-the-art database fuzzers—SQLsmith and SQLancer.
— source