A Comparative Survey of Mutation Testing Across Large Language Models, REST APIs, and Database Engines

B. Falah, Hayat Routaib. 2025. IEEE Annual Information Technology, Electronics and Mobile Communication Conference.

Read the paper · doi:10.1109/iemcon67450.2025.11381190

What this paper does with SQLancer

SQLancer appears as one of the representative database engine tools surveyed, and query-plan-guided mutation -- SQLancer's QPG -- is named as a contribution of that domain. A survey comparing how mutation testing is applied across three domains -- large language models, REST APIs and database engines -- based on 30 peer-reviewed studies. It compares mutation operator design, tool support and evaluation methods in each, reporting that LLM work is early-stage, REST API work benefits from adaptive schema-based tools, and database engine work uses grammar- and query-plan-guided mutation. Written by claude-opus-5 from the 3 places this paper refers to SQLancer. The quotations below are the paper's own words, stored verbatim when the text was extracted.

How it was classified

uses infrastructure — no

SQLancer is cited, not used; nothing in the mentions describes reusing its code.

extends technique — no

No technique is extended; the citation is background.

compares with — no

No empirical comparison against SQLancer is reported in the mentions.

describes as state of the art — no

The citation does not characterise SQLancer as the state of the art.

SQLancer publications it cites (1)

Bibliography entries that resolved to a SQLancer publication, or to a paper by one of the project's authors. A sentence citing one of these numbers is a reference to SQLancer even when it never writes the name.

#	Entry	Matched as
16	J. Ba and M. Rigger, “Testing database engines via query plan guidance,” in Proc. 45th Int. Conf. Softw. Eng. (ICSE ’23), 2023, pp. 2060–2071. doi: 10.1109/ICSE48619.2023.00174.	sqlancer publication · QPG

Every place it refers to SQLancer (3)

3 sentences, each stored verbatim from the extracted text with where it was found and how. “Citation marker” means the sentence names no tool at all and was reached through a reference number that resolved to a SQLancer publication.

Id	Sentence	Found by	Where
M1	[16] proposed query plan–guided mutation strategies, while Schafer et al.	citation marker definition	C Database Engines page 2
M2	, SQLancer, GDsmith, Squirrel) Evaluation Metrics Mutation score, test quality, fairness faults, equivalence detection Bug count, coverage (line/branch/path), spec adherence Bug count, query plan coverage, semantic correctness, performance diff Key Strengths Intelligent test generation, semantic analysis, fairness e...	name background	A Mutation Operators Used page 3
M3	TABLE II EXAMPLES OF TOOLS AND UNIQUE CONTRIBUTIONS System Type Representative Tools Notable Contributions LLMPRIMG, GenFair, LLMorpheus, EvoGPT Mutant prioritization, fairness detection, iterative reasoning REST API RESTler, EvoMaster, Dredd, CTGAN Schema-compliant mutation, GAN-based fuzzing, whitebox-LLM fusion D...	name background	B Tools and Frameworks page 4

This page is rendered from _data/papers/paper_doi_10_1109_iemcon67450_2025_11381190.json, extracted from supplied pdf. 6 pages, 21 references parsed.