Finding bugs in Gremlin-based graph database systems via Randomized differential testing

← Research building on SQLancer

Yingying Zheng, Wensheng Dou, Yicheng Wang, Zheng Qin, Leile Tang, Yu Gao, Dong Wang, Wei Wang, Jun Wei. 2022. International Symposium on Software Testing and Analysis.

Read the paper · doi:10.1145/3533767.3534409

What this paper does with SQLancer

SQLancer's three oracles are the relational precedent Grand carries into graph databases, each described in its related work. The reuse goes further than the text says: Grand's repository holds SQLancer's own source under a renamed package, which is the only evidence of it. Grand finds logic bugs in Gremlin-based graph database systems by constructing semantically equivalent databases across several systems and comparing the results of the same query on each; differing results point to a logic bug. It adds model-based query generation, producing valid queries likely to return non-empty results, and a data mapping approach that unifies result formats across systems. Across six widely used graph systems including Neo4j and HugeGraph it found 21 previously unknown logic bugs, 18 confirmed. Written by claude-opus-5 from the 7 places this paper refers to SQLancer. The quotations below are the paper's own words, stored verbatim when the text was extracted.

How it was classified

uses infrastructure — yes (implementation)

Nothing in the text claims reuse -- SQLancer appears only in related work. The repository carries SQLancer's source under a renamed package, which is what establishes it, and is why the claim cites the artifact rather than a sentence.

Artifact https://github.com/choeoe/Grand carries: renamed_sqlancer_package, sqlancer_source_content_match ARTIFACT · artifact inspection

extends technique — uncertain

Grand applies differential testing across graph systems rather than generalising PQS, NoREC or TLP, but with the codebase reused and no statement of what was changed, the boundary is not settled by what is here.

compares with — no

SQLancer's oracles are described as relational precedent; no run against them is reported.

describes as state of the art — no

The oracles are described by what they do rather than as the state of the art.

Where this differs from the pattern checks

The regular expressions that scan for these relationships are advisory. Where the reading above contradicts one, the reason is recorded.

Its artifact

choeoe/Grand carries renamed sqlancer package, sqlancer source content match.

Repository is named after Grand, the tool this paper says it built, and is about database testing.
# Grand
source

src/main/java/org/gdbtesting/Randomly.java is SQLancer's Randomly.java, with the package renamed to org.gdbtesting (8 of 8 identifiers match: StringGenerationStrategy, SOPHISTICATED, cachedLongs, cachedStrings, cachedDoubles).
private static StringGenerationStrategy stringGenerationStrategy = StringGenerationStrategy.SOPHISTICATED;
source

SQLancer publications it cites (4)

Bibliography entries that resolved to a SQLancer publication, or to a paper by one of the project's authors. A sentence citing one of these numbers is a reference to SQLancer even when it never writes the name.

#	Entry	Matched as
13	2021.SQLancer. Retrieved August 5, 2021 from https://github.com/sqlancer/ sqlancer	sqlancer publication
48	ManuelRiggerandZhendongSu.2020. DetectingOptimizationBugsinDatabase EnginesviaNon-OptimizingReferenceEngineConstruction.In Proceedingsof ACM Joint European Software Engineering Conference and Symposium on the Foundati...	sqlancer publication · NOREC
49	Manuel Rigger and Zhendong Su. 2020. Finding Bugs in Database Systems via Query Partitioning. Proceedings of the ACM on Programming Languages 4, OOPSLA, Article211(2020), 30pages.	sqlancer publication · TLP
50	ManuelRiggerandZhendongSu.2020. TestingDatabaseEnginesviaPivoted Query Synthesis. In Proceedings of USENIX Symposium on Operating Systems Designand Implementation(OSDI). 667ś682.	sqlancer publication · PQS

Every place it refers to SQLancer (7)

7 sentences, each stored verbatim from the extracted text with where it was found and how. “Citation marker” means the sentence names no tool at all and was reached through a reference number that resolved to a SQLancer publication.

Id	Sentence	Found by	Where
M1	For relational database systems that utilize Structured Query Language (SQL) to create, access, and modify data, researchers havedevelopedseveraltools,suchas RAGS[ 52]and SQLancer[ 13], toeffectivelydiscoverlogicbugs.	citation marker background	1 INTRODUCTION page 2
M2	, Pivoted Query Synthesis (PQS) [ 50], Ternary LogicPartitioning(TLP)[ 49],andNon-OptimizingReferenceEngine Construction (NoREC) [ 48] to find logic bugs.	technique definition	1 INTRODUCTION page 2
M3	2 Model-BasedQueryGeneration Randomquerygenerationisacommonly-usedapproachfortesting databases [ 13,14,52].	citation marker background	3.2 Model-BasedQueryGeneration page 5
M4	providetwometamorphictestingapproaches[ 48, 49], namely Ternary LogicPartitioning (TLP) and Non-optimizing ReferenceEngineConstruction(NoREC),totest DBMS.	technique definition	7.1 TestingofDBMS page 11
M5	TLP[ 49] partitions a query into three sub-queries, and detects bugs by comparing the combination of results of three sub-queries with the result of the original query.	technique definition	7.1 TestingofDBMS page 11
M6	NoREC [ 48] compares the execution results of a given optimized query with its non-optimized version, to detect optimization bugs in DBMS.	technique definition	7.1 TestingofDBMS page 11
M7	SQLanceroffersPivotedQuerySynthesis(PQS)[ 50]approachto findlogicbugsbyrandomlyselectingapivotrowasoracleandgenerating random queriescontainingthe selectedrowtotest DBMS.	technique definition	7.1 TestingofDBMS page 11

This page is rendered from _data/papers/paper_doi_10_1145_3533767_3534409.json, extracted from supplied pdf. 12 pages, 55 references parsed.