One DBMS, Two Modes, and a Bunch of Bugs: Catching Logic Bugs in Distributed DBMSs via Differential Testing

Zikun Fu, Jiaju Bai, Hong-Bo Feng, Kang Chen. 2026. Proceedings of the ACM on Management of Data.

What this paper does with SQLancer

QPG is DistSQL's closest comparison and its point of departure. The paper describes QPG as improving SQLancer by using query plan text as feedback to mutate the database state when no new plans appear, and argues this focuses on centralized database state and so becomes noisy and less informative on distributed systems. That claim is tested directly: one of DistSQL's own ablation variants uses query-plan-text feedback identical to QPG's, and QPG is also run as an external baseline alongside Jepsen, Mallory and EET, with DistSQL finding 28 more bugs. NoREC, TLP, PQS and CODDTest appear as the local-DBMS oracles the distributed setting is contrasted with. DistSQL tests distributed DBMSs by running the same query in the system's distributed and single-node modes and comparing the results, so a difference between the two modes is a bug without needing to know the right answer. Its feedback signal is drawn from query plan features rather than plan text, which the paper argues is necessary because a distributed plan's text is noisy. Across five open-source distributed DBMSs it found substantially more bugs than the existing tools. Written by claude-opus-5 from the 18 places this paper refers to SQLancer. The quotations below are the paper's own words, stored verbatim when the text was extracted.

How it was classified

uses infrastructure — no

M8 records an AST-based SQL generator of the authors' own. QPG is run as a baseline and reimplemented as an ablation variant, neither of which makes DistSQL built on SQLancer.

extends technique — no

DistSQL replaces plan-text feedback with plan-feature feedback, which M7 presents as a correction of QPG's behaviour on distributed systems rather than a generalisation of it, and its oracle is differential execution across two modes.

compares with — yes

M10 lists QPG among the state-of-the-art tools DistSQL is compared against, M12 records it as one of four tools run on five distributed DBMSs, and M15 gives the result: 28 more bugs than QPG.

Query Plan Guidance (QPG)

We compare DistSQL against SOTA database and distributed system testing tools, including Squirrel [ 65], QPG [ 2], and EET [ 24], as well as distributed system testing tools, including Jepsen [20] and Mallory [36]. M10 · 7.1 Experimental Setup · page 14

4 Comparison We evaluate five open-source distributed DBMSs (Table 3) using four existing tools: Jepsen [ 20], Mallory [36], QPG [2], and EET [24], and compare their performance with DistSQL. M12 · 7.4 Comparison · page 17

DistSQL identifies 28 more bugs than QPG and 31 more than EET. M15 · 7.4 Comparison · page 17

describes as state of the art — yes

M10 describes QPG as one of the SOTA database and distributed system testing tools DistSQL is measured against.

SQLancer publications it cites (8)

Bibliography entries that resolved to a SQLancer publication, or to a paper by one of the project's authors. A sentence citing one of these numbers is a reference to SQLancer even when it never writes the name.

#	Entry	Matched as
2	Jinsheng Ba and Manuel Rigger. 2023. Testing database engines via query plan guidance. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 2060–2071. 4, No. 1 (SIGMOD), Article 59. Pub...	sqlancer publication · QPG
3	Jinsheng Ba and Manuel Rigger. 2024. Keep It Simple: Testing Databases via Differential Query Plans. Proceedings of the ACM on Management of Data 2, 3 (2024), 1–26.	sqlancer publication · DQP
23	Zu-Ming Jiang, Si Liu, Manuel Rigger, and Zhendong Su. 2023. Detecting transactional bugs in database engines via {graph-based}oracle construction. In 17th USENIX Symposium on Operating Systems Design and Implementati...	project authored
43	Manuel Rigger and Zhendong Su. 2020. Detecting optimization bugs in database engines via non-optimizing reference engine construction. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conf...	sqlancer publication · NOREC
44	Manuel Rigger and Zhendong Su. 2020. Finding bugs in database systems via query partitioning. Proceedings of the ACM on Programming Languages 4, OOPSLA (2020), 1–30.	sqlancer publication · TLP
45	Manuel Rigger and Zhendong Su. 2020. Testing database engines via pivoted query synthesis. In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20). 667–682.	sqlancer publication · PQS
50	sqlancer. 2020. Automated testing to find logic and performance bugs in database systems. https://github.com/ sqlancer/sqlancer.	sqlancer publication
61	Chi Zhang and Manuel Rigger. 2025. Constant Optimization Driven Database System Testing. Proceedings of the ACM on Management of Data 3, 1 (2025), 1–24.	sqlancer publication · CODDTEST

Every place it refers to SQLancer (18)

18 sentences, each stored verbatim from the extracted text with where it was found and how. “Citation marker” means the sentence names no tool at all and was reached through a reference number that resolved to a SQLancer publication.

Id	Sentence	Found by	Where
M1	Some approaches check the consistency of semantically equivalent predicates or expressions across multiple statements [ 2,18,24,44,45,48,61].	citation marker background	1 Introduction page 2
M2	Others verify the correctness of query optimization [ 43,53] or specific features such as indexes and transactions [ 3,23,30,49].	citation marker background	1 Introduction page 2
M3	Approach Main Target Bug Type Feedback Black-Box Logic Checker Database State Mutation Morpheus [58] Distributed System Concurrency Conflict Analysis No-Oathkeeper [33] Distributed System Semantic None No-Chronos [8] Distributed System Timeout None No-Generic Jepsen [20] Distributed System Logic None Yes Fixed Rules...	technique baseline	1 Introduction page 4
M4	EET [ 24], and CODDTest [ 61] find logic bugs by rewriting queries into another forms that are expected to produce the same results.	technique definition	2.2 Limitations of Existing Approaches page 5
M5	DQP [ 3] compares executions with different query hints applied, where query plans act as local heuristics rather than guiding test case generation.	technique definition	2.2 Limitations of Existing Approaches page 5
M6	QPG [ 2] improves SQLancer [ 50] by using query plan text as feedback to mutate the database state when no new plans are found, which focuses only on centralized database states and ignores distributed ones.	name definition	2.2 Limitations of Existing Approaches page 6
M7	Consequently, QPG is less effective on distributed DBMSs, as its feedback signal becomes noisy and less informative.	technique motivation	2.2 Limitations of Existing Approaches page 6
M8	We implement an abstract syntax tree (AST) based SQL generator [ 1,22,45,47] that constructs queries in a top down manner by first selecting the SQL statement type (e.	citation marker background	3.2 Testing Workflow page 7
M9	One approach is to treat the entire query plan as a pattern [ 2].	citation marker definition	5.3 Plan Patterns based on Operator Interactions page 10
M10	We compare DistSQL against SOTA database and distributed system testing tools, including Squirrel [ 65], QPG [ 2], and EET [ 24], as well as distributed system testing tools, including Jepsen [20] and Mallory [36].	technique state of the art	7.1 Experimental Setup page 14
M11	3 Sensitivity Analysis To evaluate the effectiveness of our feedback mechanism based on query plan feature, we implement three variants of DistSQL: (1) DistSQL Random, which generates random SQL queries without feedback, (2) DistSQL Text, which uses the query plan text-guided feedback identical to that of QPG [ 2], ...	technique baseline	7.3 Sensitivity Analysis page 16
M12	4 Comparison We evaluate five open-source distributed DBMSs (Table 3) using four existing tools: Jepsen [ 20], Mallory [36], QPG [2], and EET [24], and compare their performance with DistSQL.	technique baseline	7.4 Comparison page 17
M13	Among recent DBMS testing tools, QPG employs feedback mechanism related to query plan, while EET achieves higher SQL diversity than many prior approaches [ 24].	technique definition	7.4 Comparison page 17
M14	It also surpasses Jepsen, Mallory, and QPG by orders of magnitude in both metrics.	technique result comparison	7.4 Comparison page 17
M15	DistSQL identifies 28 more bugs than QPG and 31 more than EET.	technique result comparison	7.4 Comparison page 17
M16	Other approaches [ 2,18, 23,24,30,43–45,48,53,61] aim to find logic bugs by validating the query processing logic in local DBMSs through test oracles, which typically compare the execution results of two logically equivalent or related SQL queries.	citation marker background	9.2 DBMS Fuzzing page 20
M17	Other approaches [ 3,29,49] adopt differential testing strategies, executing the same SQL statements under different runtime configurations or query hints to identify inconsistencies.	citation marker background	9.2 DBMS Fuzzing page 20
M18	One DBMS, Two Modes, and a Bunch of Bugs: Catching Logic Bugs in Distributed DBMSs via Differential Testing 59:21 feedback signals, such as formal rule coverage in SemConT [ 31] and graph similarity of test cases in TQS [ 53], while others adopt more generic runtime feedback, such as branch coverage [ 30] or query p...	citation marker	9.2 DBMS Fuzzing page 20

This page is rendered from _data/papers/paper_doi_10_1145_3786673.json, extracted from supplied pdf. 24 pages, 65 references parsed.