SEOUL, September 04 (AJP) - When an artificial intelligence system writes a database query that will not run, the database that rejected it already knows what is wrong with it. Most systems treat that rejection as a plain failure signal and hand the whole query back to the AI to be written again from scratch. A team at the Korea Advanced Institute of Science and Technology has built software that reads the rejection as a set of directions and repairs only the broken piece.
Tested on BIRD, a benchmark researchers use to compare systems of this kind, the software cleared up to 87.4 percent of the execution errors in queries the AI produced on its first attempt, and raised execution accuracy, the share of questions that come back with the right answer, by up to 5.8 percentage points over the strongest previous method. Measured against systems that regenerate the query from scratch, it consumed up to 15.1 times fewer tokens, the units of text a language model is billed by, and finished its repairs up to 29.6 times faster.
The software is called SafeQL. It was built at the Korea Advanced Institute of Science and Technology (KAIST), South Korea's state-funded science and technology university, by a School of Computing team led by Kim Min-soo. Doctoral student Lee Geon-ho was the first author on the paper, and Kim was the corresponding author. The work was published in the Proceedings of the VLDB Endowment and listed in the program of the International Conference on Very Large Data Bases, which ran from Aug. 31 to Sept. 4 in Boston.
Databases answer only in SQL, short for Structured Query Language, a command language that requires every table and column to be named exactly as it exists in the system. Text-to-SQL tools remove that requirement. A manager can ask which product sold best last year, or which items are running low in stock, and the AI converts the question into SQL and returns the figures.
The conversion goes wrong often enough to matter. A language model querying a company's sales records may invent a column that does not exist, call a function the database does not support, or link two tables in a way the underlying structure does not permit. The query fails to execute, and the AI comes back with nothing.
Until now, the standard remedy has been to send the error message back to the model and ask for a replacement. Rewriting an entire document because one word in it is wrong carries an obvious cost, and query repair works the same way. Parts that were already correct can change, new errors can appear in place of the old one, and every round trip to the model spends more tokens and more time.
SafeQL keeps the query the AI wrote and works from it. Reading the error the database reports alongside the schema it can inspect directly, the software locates the element that failed and replaces it, swapping a nonexistent column for a real one, adding a missing table link, or correcting a function or a search value.
A single error usually has several plausible fixes, and testing all of them across a chain of repairs would multiply the candidates beyond any useful limit. KAIST defined the set of queries the database can actually execute as a safe query space, then searched toward it by trying first the candidates closest in structure and meaning to what the model originally wrote. Candidates whose data types do not match are discarded before they are tested. Kim and Lee proved mathematically that the search arrives at the safe query space within a finite number of steps.
Rather than sit alongside the database, SafeQL runs inside PostgreSQL, the open source database system used across much of the software industry. Working from the parse tree PostgreSQL builds for every incoming command, it can pinpoint the failure even in long and heavily nested queries. Repeated similarity calculations are held in a cache and a vector index to keep the search cheap. When the first query is too badly broken to patch, the software calls the language model back for a fresh one and resumes the search from there.
KAIST said the approach should reduce the cost and time of running AI on company data, and bring closer the point at which AI agents can handle sales, customer and inventory records reliably enough to be trusted with the work. Errors at the data access stage carry forward, since everything an agent does next rests on figures it was unable to retrieve in the first place.
"For AI to take on real work at a company, accurately finding the data it needs is what matters," Kim said.
On Spider, an older benchmark drawn from databases across many different fields, SafeQL reached 91.7 percent execution accuracy while using up to 96.5 times fewer tokens than regeneration methods and cutting repair time by up to 34.5 times.
[Reference Information]
Title: SafeQL: Search-based Refinement for Safe and Efficient LLM-based Text-to-SQL
Link/DOI: 10.14778/3819518.3819545
AJP Takeaways
- KAIST's SafeQL repairs only the failed element of an AI-generated database query rather than regenerating the entire query, clearing up to 87.4 percent of execution errors on the BIRD benchmark.
- Kim Min-soo's team reported token use down by up to 15.1 times and repair time down by up to 29.6 times against regeneration methods, with 91.7 percent execution accuracy on the Spider benchmark.
- PostgreSQL hosts the software directly, giving it the database's own parse tree to locate failures inside long and heavily nested queries.
- The VLDB Endowment published the paper in its Proceedings, and the International Conference on Very Large Data Bases listed it in a program that ran Aug. 31 to Sept. 4 in Boston.
Copyright ⓒ Aju Press All rights reserved.
