Raw capture â raw/2604-21413v2.md
url: https://arxiv.org/html/2604.21413v2 date_fetched: 2026-08-25
Enterprise data exists in many forms, such as tables, text, maps, e-mail, and CAD models, that are invariably access-controlled and behind bespoke interfaces. Current agentic AI systems delegate the entire query workflow to a frontier LLM: a single model is left to interpret the request, decide which sources or tools to use, integrate the evidence retrieved across them, judge its own completeness, and generate an answerâwith few constraints, little utilization of schemas, and text as the primary representation throughout. We argue that this methodology is an ineffective abstraction for enterprise data, and that reliable agentic AI should instead require structure: a constrained query interface over each source and a table-centric integration layer that a query processor drives. We introduce RUBICON, a system that embodies this vision.
RUBICON is based on two fundamental observations. First, text-to-SQL fails on real enterprise data and must be dramatically subsetted to achieve reliable results. Second, data integration across disparate corporate datasets is best performed using tables as the core abstraction rather than the text-centric pipelines of current LLMs and agentic AI systems.
We evaluate RUBICON on two benchmarks: on our own enterprise-focused benchmark, RUBICON-Bench, against two agentic systems, as well as on a recent semantic query benchmark called SemBench against two state-of-the-art semantic query processing systems, LOTUS and Palimpzest. On RUBICON-Bench, where queries require coordinating across heterogeneous enterprise sources, RUBICON achieves 100% end-to-end accuracy, while all agentic baselines â including single- and multi-agent ReAct systems â produce no correct answers. On SemBench, RUBICON surpasses both LOTUS and Palimpzest: it achieves 14.7% higher accuracy, reduces end-to-end latency by 62.64%, and lowers token cost by 98.64%, demonstrating that a table-centric architecture better matches the structure of enterprise data while also yielding significant efficiency gains in enterprise settings.
â
â authors: .
PVLDB Reference Format:
PVLDB, 14(1): XXX-XXX, 2020.
doi:XX.XX/XXX.XX â â
This work is licensed under the Creative Commons BY-NC-ND 4.0 International License. Visit https://creativecommons.org/licenses/by-nc-nd/4.0/ to view a copy of this license. For any use beyond those covered by this license, obtain permission by emailing info@vldb.org. Copyright is held by the owner/author(s). Publication rights licensed to the VLDB Endowment.
Proceedings of the VLDB Endowment, Vol. 14, No. 1 ISSN 2150-8097.
doi:XX.XX/XXX.XX
PVLDB Artifact Availability:
The source code, data, and/or other artifacts have been made available at https://github.com/fabian-wenz/rubicon.
Large Language Models (LLMs) were all the rage three years ago as researchers tried to figure out what to do with them. It quickly became apparent that LLMs needed to be augmented by RAG, fine tuning, prompt engineering, etc. to achieve good results. Hence, LLMs quickly evolved into agentic AI, a pipeline (or graph) of processing steps centered around an LLM. Agentic AI requires an orchestrator to decide on a sequence of actions to take, and traditional agentic AI entrusts this step to an LLM (Urban and Binnig, 2024; Eckmann and Binnig, 2026).
We believe there are two fundamental problems with this architecture in an enterprise setting.
First, one needs to access much corporate data through SQL interfaces. Although LLMs do reasonably well on public text-to-SQL benchmarks, such as Spider (Yu et al., 2018) and Bird (Li et al., 2024a) where accuracies above 80% (Li et al., 2024b) are reported, they fail miserably on real-world inspired benchmarks such as Beaver (Chen et al., 2024) (accuracy around 10%). Accuracy on Beaver rises to 30+% when the prompt includes the tables in the from clause and all the join terms. The reasons are detailed in (Chen et al., 2024) and include:
Hence, full SQL is nearly guaranteed to generate failure in an agentic AI environment. As a result, one requires "baby SQL", for which we propose Agentic Query Language (AQL).
The second reason concerns the mechanism for data integration. Using the text representation of a typical LLM as the integration framework will cost a lot of money (tokens) and produce poor accuracy. For example, an insurance underwriting application must integrate maps (say of flood zones), CAD (building drawings), previous claims (tables), and underwriting notes (text). It is very unlikely that converting all these sources to text so an LLM can do the integration will succeed. A much better idea is to convert the data sources to tables. This will require a table orchestrator, in other words a relational query optimizer.
For these two reasons, we propose an alternate agentic AI architecture, well-suited to enterprise data. It leverages three main concepts:
AQL as a mechanism to access corporate data. This requires tabular orchestration to generate complex SQL commands and access to data behind bespoke interfaces using adaptors to mediate the conversion.
Access to text through LLMs, embedding or inverted indexes and conversion of the resulting documents to tables for ingestion by a query optimizer.
A human in the loop. Since searches may go "off the rails" we need the ability to inspect intermediate results and correct the plan. A human in the loop is also required to help orchestrate complex queries.
The remainder of this paper discusses AQL and the architecture of RUBICON. We then conclude with benchmark results comparing RUBICON to other approaches on a benchmark inspired by a real application on which we are collaborating with the Department of Transportation of the City of Munich (Germany).
RUBICON has three moving parts: AQL, wrappers, and intermediate results. AQL is the user-facing notation. Wrappers are the source-facing translation layer. Intermediate results are the system-facing representation that makes execution inspectable and optimizable.
The core abstraction is Agentic Query Language (AQL), a restricted SQL-like notation that is a compromise between natural language and SQL. Natural language is easy to write, but has unacceptably bad accuracy in enterprise settings. Full SQL has the opposite problem. It is explicit, optimizable, and reproducible, but requires users to know the schema and write precise queries. AQL is a compromise between these two extremes.
The basic AQL form is:
FIND
FROM
WHERE
The user specifies the data source, relation, and desired attributes explicitly; otherwise there is little hope for acceptable accuracy. The predicate may remain in natural language. AQL also supports different degrees of explicitness. For example, a novice may write:
FIND professors
FROM UniversityDW
WHERE faculty members affiliated with the research lab
whereas an expert may write:
FIND Faculty.name, Faculty.title
FROM UniversityDW.Faculty
WHERE Faculty.affiliation = âResearch Labâ
AND Faculty.rank IN (âProfessorâ, âFull Professorâ)
Both queries produce the same logical result object:
ResearchLabProfessors(name, title)
The difference is not the desired answer, but how much structure the user chooses to expose. The more structure is specified, the less work is delegated to the LLM or wrapper.
AQL also supports joins by composing two source-local queries:
FIND name, title
FROM UniversityDW.Faculty
WHERE faculty members affiliated with the research lab
JOIN
FIND name, award
FROM Wikipedia.Pages
WHERE people who have won a Turing Award or Nobel Prize
The important point is not syntactic novelty; rather it is allocation of responsibility. The human or interface specifies the structural part of the query: sources, fields, and intermediate objects. The wrapper or LLM handles only the local ambiguity inside the predicate. This makes LLM use narrow, inspectable, and replaceable.
AQL also includes minimal schema-inspection and housekeeping commands. The command ? lists available sources, ? lists available relations in a source, and ? lists attributes. Intermediate results can be named using SAVE. These commands are deliberately mundane. Their purpose is to make exploration explicit rather than conversational.
As a result. a novice can start with mostly natural predicates and a GUI that helps select sources and fields. An expert may write more explicit AQL, approaching SQL. In both cases, the result is not a hidden chain of tool calls but a visible query object that can be saved, modified, re-executed, and optimized. Of course, it is possible to give RUBICON the entire user task and ask it to compile the task into a ordered collection of AQL tasks. As mentioned earlier, we do not expect RUBICON to produce good results in this mode.
RUBICON treats retrieval as a query processing problem. It is closer in spirit to mediator-wrapper data integration than to RAG-style ingestion: sources remain in place, wrappers expose logical relations, and integration occurs at query time.
Many enterprise copilots follow an ingestion-and-indexing architecture: they connect to enterprise systems, extract documents or records, compute embeddings or sparse indices, and expose a unified semantic search interface. This improves discoverability, but it does not integrate the underlying data models. A text index is not a join engine. It is useful for finding evidence; it is not a substitute for an execution plan.
RUBICON instead follows the classical database approach: leave data sources in place, put a wrapper over each source, and perform integration at query time. A wrapper translates AQL into the native interface of the underlying source, enforces source-local access control, and normalizes the returned result into a logical relation. The source may be a database, a search engine, an e-mail system, a web API, or an LLM. The query processor should not care.
For example, an e-mail system can expose a logical table:
Message(from, to, subject, date, body, thread_id)
A Wikipedia wrapper can expose:
Page(title, url, snippet, categories, text)
A data warehouse already exposes relations, while an LLM wrapper can expose its answer as a document relation with text, metadata, and provenance. These tables need not be materialized relations. They are logical views constructed by wrappers. This normalization step is essential: without a tabular representation downstream operators such as joins, unions, and filters will fail.
Execution proceeds in two modes. In interactive mode, the user issues one AQL command, inspects the result, and then proceeds. Each command materializes an intermediate relation that can be inspected, saved, reused, or joined with later results. This mode is appropriate for exploratory enterprise questions, where the right plan is often discovered step by step.
In compiled mode, a sequence of AQL commands is executed as a single composite plan. Once the system has the full command sequence, it can apply database-style optimization: predicate pushdown, join reordering, source selection, and cost-aware execution. Costs may include not only runtime, but also tool calls, latency, and token consumption. In this sense, retrieval is not the step before query processing. Retrieval is one operator inside query processing.
We have been working with the Department of Transportation of the City of Munich (Germany). They have four full time engineers answering citizen complaints/queries, often of the form "I donât have time to cross the street before the light turns." or "Why is the trolley always late?" These engineers consult the following data sources to address such concerns: light schedule (tables), trolley schedule (tables), diagrams of intersections (CAD), and Germany and Munich laws (text). They have tried agentic AI with dismal results and have found preliminary RUBICON results encouraging. Unfortunately, it will be a long process to get permission to anonymize and make public their data and internal benchmarks.
In the meantime, we have constructed RUBICON-Bench, which is similar to the Munich problem. This benchmark models a university-style enterprise environment in which relevant information is distributed across five heterogeneous sources: an institutional data warehouse, a research laboratory website, Wikipedia/API access, e-mail, and LLM knowledge. The data warehouse contains structured administrative information such as people, buildings, rooms, and organizational affiliations. The lab website contains semi-structured public information about research groups, events, projects, and personnel. E-mail represents private communication, while Wikipedia and the LLM source represent external public knowledge.
The workload contains seven natural-language queries. Each query requires coordination across two relevant sources, while the remaining sources act as distractors. Representative queries include:
Which CSAIL professors have won a Turing Award or a Nobel Prize?
What research lab events have taken place in a specific campus building over the past month?
These queries are deliberately simple to read but nontrivial to execute. For example, the question How many university buildings have a Wikipedia page? requires retrieving the set of buildings from the university data warehouse, retrieving candidate pages from Wikipedia, resolving name variants, and counting the joined result. Similarly, the question Which research lab professors have won a Turing Award or a Nobel Prize? requires identifying research lab professors from the institutional source and joining them with external award evidence. In both cases the system must use the correct sources and make the cross-source join explicit.
| System | Query | ($) | Latency(s) | |||| | Vanilla | NL | 1 105.57 | 0.0 | 0.0016 | 13.70 | 0.00% | 0.00% | | Vanilla | AQL | 1 447.43 | 0.0 | 0.0021 | 12.80 | 0.00% | 0.00% | | Single-Agent | NL | 7 609.43 | 2.14 | 0.0180 | 16.07 | 0.00% | 0.00% | | Single-Agent | AQL | 36 986.00 | 4.57 | 0.0754 | 25.75 | 0.00% | 71.43% | | Multi-Agent | NL | 34 772.43 | 10.00 | 0.0800 | 47.94 | 0.00% | 14.29% | | Multi-Agent | AQL | 8 672.14 | 3.14 | 0.0324 | 28.72 | 28.57% | 100.00% | | RUBICON | NL | 2 440.29 | 2.00 | 0.0048 | 6.99 | 100.00% | 100.00% | | RUBICON | AQL | 242.29 | 2.00 | 0.0011 | 4.37 | 100.00% | 100.00% |
Note. All runs in this table use OpenAIâs GPT-5.2.
Table 1 reports the results of running RUBICON-Bench on the execution modes considered in this paper. Each of 4 systems is run in two different modes. In the natural-language (NL) variant, the system is given only the user question and must decide for itself which sources to access and how to combine them. In the AQL-guided variant, the input is an explicit query plan that names the sources and specifies the intended multi-source execution plan.
The first system is a vanilla LLM (OpenAIâs GPT 5.2), and the two variants are supported through prompt engineering. The next rows show a Single-Agent ReAct(Yao et al., 2023) system which is a single LangChain11 1
https://github.com/langchain-ai/langchain ReAct agent with five tools (SQLite, Gmail, Wikipedia, GPT, and CSAIL). Given a natural-language question, the agent freely decides which tools to call and in what order through its own reasoning loop. The AQL guided variant keeps the exact same agent and ReAct loop but reformats the input as an explicit AQL JOIN plan (naming the two sources and what to retrieve from each).
The third set of rows shows the Multi-Agent variant, namely a LangGraph22 2
https://github.com/langchain-ai/langgraph StateGraph where the orchestrator itself is the ReAct loop(Yao et al., 2023). It reasons ("what do I need next?"), acts (calls one sub-agent), observes the result (inspects collected results), and reasons again before deciding the next step â up to four times.
Each sub-agent is a single-tool ReAct loop restricted to one data source. The AQL-guided variant uses the same graph but passes the pre-written AQL plan as an instruction, telling the orchestrator exactly which two agents to call and in what order, effectively constraining its iterative reasoning to match the pre-specified plan.
The last option is RUBICON. It is evaluated in the same two input modes used throughout the paper. As explained in Section 2, NL mode corresponds to a novice natural-language request, while AQL mode corresponds to a more explicit expert specification.
We report two accuracy metrics in Table 1. is end-to-end execution accuracy. A query receives score 1 only if the final answer is correct and 0 otherwise. There is no partial credit for calling one of the right sources or making progress toward the answer. On this metric, only RUBICON performs well.
is a source-selection metric. It asks a simpler question: did the system call the correct sources, tools, or agents? The AQL variants do well on , largely by construction, since the plan specifies the required sources. The NL variants do poorly because the LLM planner must infer the source set from the question alone.
The remaining columns report aggregate efficiency metrics averaged over the benchmark. is the mean number of tokens per query, is the mean number of tool calls, is the mean provider-reported cost, and latency is the mean time to first answer. These metrics show that RUBICON has lower latency than the agentic baselines and is generally a great deal cheaper.
The results support several observations. Vanilla LLM scores 0% on all queries in both modes. Even an explicit AQL sequence does not help. Its single one-shot operation naturally causes low token numbers, cost, and latency. On the other hand, agentic methods result in each additional reasoning step being another LLM call thereby increasing tokens, cost, and latency.
Among the agentic baselines, the Single-Agent NL follows a narrow linear chain. It tries to find the answer by successively narrowing the scope, which keeps cost low but usually misses a second data source, if one exists. Of course, Single-Agent AQL fixes the source-selection problem by specifying both sources explicitly. The price is two full ReAct loops plus a synthesis step, which yields roughly five times as many tokens. A Multi-Agent NL has an orchestrator sees the full source landscape which explores broadly, exhausting its call budget on every query and producing the highest token count and cost. Multi-Agent AQL restricts execution to the required sources, cutting both tokens and latency.
remains near zero for all agent variants. The reason is simple: the sub-agents return prose, not relations. Hence the final âjoinâ is a join over unstructured text, which language models do unreliably. RUBICON, on the other hand, retrieves relational tables and applies explicit joins. As a result, RUBICON is the clear winner across all metrics. Its AQL variant uses the fewest tokens, completes fastest, and achieves 100% on both and . The NL variant is, of course, more expensive because of the cost of NL. Even so, it is still cheaper and faster than any agentic baseline, while achieving perfect accuracy.
We also ran RUBICON on SemBench (Lao et al., 2025), another multimodal query scenario of similar complexity to RUBICON-Bench. Notice that RUBICON-Bench is focused on whether a system can identify the correct data sources join the output and determine that the answer is complete. In contrast, SemBench tests cost and accuracy of fixed pre-specified plans.
We evaluated the SemBench E-Commerce scenario. The workload is one logical source, a fashion dataset, exposed through three access paths: structured attributes, text embeddings, and image embeddings. The joins are therefore intra-catalog joins, not joins across independent enterprise systems. As a result, it is much simpler than Rubicon-Bench.
We executed the workload in the intended RUBICON style: source-local Find operations produce intermediate tables, and subsequent operators join these tables explicitly. This simple design works well. RUBICON achieves an average quality of 0.86 at a cost of $0.003 and latency of 55.74 s. On the same scenario, SemBench reports 0.75 quality, $0.22 cost, and 149.2 s latency for LOTUS, and 0.70 quality, $0.42 cost, and 192.5 s latency for Palimpzest (Lao et al., 2025). For this single-source, multi-modal workload, treating semantic predicates as explicit operators and joining materialized intermediate results is cheaper and more robust than pushing the whole task through an LLM-driven execution engine.
This work has evaluated RUBICON against traditional LLMs and agentic AI. The basic question is whether a less constrained interface can perform well using increased reasoning effort and broader tool access on multi-source coordination retrieval problems. Across all configurations, a consistent pattern emerges: greater autonomy does not translate into correct multi-source integration. Even when equipped with full tool access and high reasoning effort, models frequently fail to identify all required sources, invoke them in the appropriate order, or enforce completeness before termination.
Unless (or until) this problem is rectified, one must use a tool like RUBICON. In addition, one can imagine using a hybrid system where all structured data sources are serviced by something like RUBICON. Then traditional agentic AI performed the integration between text and tables. As a next step, we will compare such a system with RUBICON, but we are skeptical that it will work better. Discarding the structure in structured data so it can be combined with text in a text-oriented system just throws away semantics. Its hard to believe that this is a good idea. Furthermore, entrusting integration to an agentic AI system increases token consumption and increases cost.
Overall, the results indicate that LLM-centric coordination does not work well on enterprise-grade multi-source problems. Utilizing RUBICONprovides more robust results its human-in-the-loop database-oriented architecture.