Skip to main content

Command Palette

Search for a command to run...

APEX AI Agents: Comparing Cost, Quality, and Models

Updated
18 min readView as Markdown
APEX AI Agents: Comparing Cost, Quality, and Models
J
Hi, thanks for stopping by! I am focused on designing and building innovative solutions using AI, the Oracle Database, Oracle APEX, and Oracle REST Data Services (ORDS). I hope you enjoy my blog.

Introduction

Are you excited about AI Agents in Oracle APEX 26.1? I am. But before we make an agent out of everything, we should understand what it costs, what actually drives that cost, and which optimizations preserve quality instead of merely making a bad answer cheaper.

To find out, I ran 69 user turns across five scripted scenarios and three models using an APEX AI Agent in Project Genie, a ("Jira-like" / "Jira-lite") project and task management app we built in Oracle APEX. The tests covered search, retrieval, synthesis, task creation, field updates, status changes, comments, verification, and user approvals. I recorded every provider exchange, token count, tool call, tool result, and elapsed-time measurement.

The short version is this:

  • Model price matters enormously, but the cheapest successful model matters more than the cheapest model.

  • Input context (system prompts, tools, etc) accounted for 91% to 93% of estimated model cost when calculated at standard uncached token rates. Output length was not the main cost driver.

  • Tool definitions are sent with each provider request, while retained tool results become part of the conversation history. Both contribute to repeated input-token usage.

  • More capable models can use tools more efficiently, but the reduction in tokens may not offset a higher per-token price.

  • For this agent and test suite, GPT-5.6 Terra was the sweet spot. It passed the same five tests as GPT-5.6 Sol at 46.5% lower estimated cost.

All costs below are estimates calculated from recorded token usage and published standard uncached rates. They exclude cache discounts and cache-write charges, so actual billed costs and the percentage attributable to input may differ. The best place to capture 100% accurate costs is by creating a separate API key and tracking the cost from the website of your LLM provider.

The agent under test

Oracle APEX 26.1 AI Agents combine a Generative AI Service, system prompt, and AI Tools. On-demand tools let the model retrieve application data or run allowed actions, while APEX manages the interaction between the model and those tools.

The Genie Agent works within the user's current workspace and permissions. It can identify accessible projects and users, search task text, comments, and subtasks, retrieve full task details, create and edit tasks, transition statuses, add comments, and open a task.

The tested agent exposed 12 on-demand tools on every provider exchange:

  • one client-side navigation tool to open a task;

  • retrieval tools for projects, users, statuses, searches, task lists, and task details; and

  • action tools for task creation, field changes, transitions, and comments.

The system prompt also specified important operational rules: stay inside the current workspace, never invent records or IDs, preserve unrequested fields, confirm bulk changes, respect action order, reuse existing evidence when safe, and treat tool-reported failures as authoritative. Oracle includes an 'extra' system prompt with every API call which is about 1,200 charcters.

Important: Security instructions in the system prompt can help the model explain access restrictions, but they are not a security boundary. Enforce access through APEX authorization schemes, database controls, and server-side checks in your tools.

That is what makes this a useful cost test. It is not a one-shot chatbot answering trivia. It is a practical application agent with authorization boundaries, database-backed tools, multi-step work, and user-approved writes.

Testing methodology

The test was run with two projects: DWEB and DOPS. The tests contained known positive matches, near matches, discussion-only matches, subtask-only matches, exact phrases, deliberate no-match cases, short descriptions, and long acceptance criteria. Before testing each model, I restored the original dataset, including removing tasks and comments created during earlier runs. Each model started with the same application data.

Each model ran the same five tests:

Test User messages per model What it exercised
T01 5 Orientation, memberships, filtered search, exact phrases, discussion evidence, and a deliberate no-match
T02 3 A short description, a long description, and a comparison using conversation history
T03 3 A broad summary, four full records, and evidence-backed synthesis
T04 7 Create, retrieve, open, edit, transition, comment, verify, and restore one task
T05 5 Resolve two exact tasks, confirm a bulk change, verify it, confirm reversal, and verify restoration

I started a fresh agent conversation for every test, but kept all messages for that test in the same conversation. This isolated the five workloads while still allowing context to accumulate inside each one. The complete suite was 23 user messages per model, or 69 messages total.

The three models tested were:

Slot Model Reasoning effort Input price per 1M tokens Output price per 1M tokens
M1 GPT-5.6 Luna none $0.20 $1.20
M2 GPT-5.6 Terra none $2.00 $12.00
M3 GPT-5.6 Sol none $4.00 $20.00

I set reasoning effort to none using the Generative AI Service’s Additional Attributes setting:

{"reasoning_effort": "none"}

Those were the published standard text rates on September 5, 2026. Sol's published rate was promotional through at least November 21, 2026, so current pricing should always be checked before using these numbers for a budget.

Only the modelchanged between slots. I verified this in the captured requests: the 12-tool JSON was identical across representative Luna, Terra, and Sol exchanges, and the normalized system-prompt text was identical.

Each expected behavior was manually checked against known facts and persisted application state. A scenario passed only if every required check passed; one failed check failed the entire scenario. “Tests passed” therefore counts successful scenarios out of five, not correct user turns out of 23.

I used custom application logging (utilizing APEX AI Agent Handlers) to record each user turn, the provider requests and responses within it, token usage, elapsed time, and tool arguments and results. This provided the measurements used in the analysis below.

The cost estimate was:

(input tokens × input rate + output tokens × output rate) / 1,000,000

The estimate does not apply cached-input discounts, prompt-cache write charges, batch rates, tax, or invoice reconciliation. No request approached the 272,000-input-token price tier: the largest observed provider request was 12,473 input tokens.

The test runbook and execution ledger are available in this GitHub Gist, with the analysis SQL available separately

Results at a glance

Model Tests passed Provider exchanges Tool calls Input tokens Output tokens Estimated cost Provider time Total run time
GPT-5.6 Luna 3/5 63 50 338,355 5,385 $0.0741 98.4 s 250.3 s
GPT-5.6 Terra 5/5 68 57 378,481 5,286 $0.8204 97.0 s 171.8 s
GPT-5.6 Sol 5/5 65 56 358,489 5,030 $1.5346 135.9 s 212.8 s

Across all three models, the test generated:

  • 69 user-visible turns;

  • 196 provider exchanges;

  • 163 tool calls;

  • 1,091,026 total tokens; and

  • $2.4291 in estimated model charges.

Every run was technically COMPLETE, token usage was recorded for every provider exchange, and every tool result was captured. Yet two Luna tests failed their functional test. This is an important observability lesson by itself: an API response and application correctness are different metrics.

The cost by scenario was:

Test Luna Terra Sol
T01 search and investigation $0.0191 $0.1856 $0.4266
T02 short/long context and reuse $0.0057 $0.0894 $0.1297
T03 multi-record synthesis $0.0113 $0.1094 $0.1559
T04 create, update, verify, restore $0.0264 $0.3197 $0.5903
T05 confirm and reverse a bulk change $0.0118 $0.1163 $0.2320

T04 was the most expensive test for every model. It had the most user turns, repeated reads after writes, several action tools, and approval cycles. The result is intuitive, but the logs show why: a business workflow costs more not merely because it calls more PL/SQL, but because every tool round can lead to another model request with a larger history.

Model choice changed both cost and quality

Luna was extraordinarily inexpensive: its complete 23-message suite cost about 7.4 cents. That was 91.0% below Terra. Luna passed the short/long-description test and both write-heavy workflows, but it failed the two tests with the most demanding retrieval and synthesis requirements.

In T01, Luna announced seven keyword matches while listing six, then corrected itself. More importantly, it made only the broad ALL search call and did not issue the required PHRASE search before reporting the exact-phrase result. In T03, it mislabeled a seven-item list as eight, exceeded the agreed word-limit tolerance, and attributed a retry behavior to the wrong task.

Terra and Sol both passed all five tests. Sol therefore delivered no measured quality improvement over Terra in this suite, even though it cost $1.5346 versus Terra's $0.8204. Terra was 46.5% cheaper than Sol.

This does not prove that Terra and Sol have equal capability. The benchmark ran each test once per model, not as a statistically repeated evaluation, and harder workloads may separate them. It does show that paying for the largest model did not improve this agent's observed result.

Luna remains attractive for tightly bounded, low-risk work or high-volume tasks with independent validation. A production routing strategy could start simple requests on Luna, use Terra as the default for mixed retrieval and action workloads, and reserve Sol for cases that are demonstrably too difficult for Terra. The key word is demonstrably: model escalation should be based on evaluations, not prestige.

Input context was the real cost center

The agent produced only 15,701 output tokens across the entire benchmark, compared with 1,075,325 input tokens. Input accounted for 91.3% of Luna's estimated charge, 92.3% of Terra's, and 93.4% of Sol's.

That changes how I think about optimization. Asking the model to be slightly more concise is useful, but it does not attack the largest part of this bill. The bigger levers are what APEX sends into every provider exchange and what conversation history is carried forward.

Even the first ordinary request in a fresh test consumed roughly 3,164 to 3,192 input tokens. As the conversations grew, the serialized message history reached 40,000 to 44,000 characters and individual provider requests reached 11,462 to 12,473 input tokens. Across all exchanges, serialized context size and input tokens had a correlation of about 0.81.

T02 makes this visible. Its third message explicitly asked the agent to use information already discussed and triggered no tools, but the provider still received 5,204 input tokens on Luna, 6,697 on Terra, and 5,379 on Sol. The model had to reread the earlier questions, answers, tool calls, and retrieved descriptions to answer from context.

T03 was more pronounced. Terra first received a 12,464-character search result, then a combined 19,001 characters across multiple full-detail tool results. Its final synthesis turn called no tool but still consumed 11,292 input tokens because the evidence was already in the conversation.

Context reuse avoids another database call, but it is not free. It trades tool execution for repeated model input. The right design depends on which representation is smaller: the retained history, a fresh compact summary, or a new targeted lookup.

Practical controls include starting a new conversation when the task changes, returning summaries before details, fetching only requested fields, and avoiding large records when an ID plus a compact projection will do.

The system prompt and tool definitions are part of every request

The captured system prompt was about 2,900 serialized characters. More surprising was the tool block. All 12 tools were offered on every exchange. Its compact captured representation was 16,192 characters.

Every provider request included one current set of the 12 tool definitions. These definitions did not accumulate another copy with each user turn. Instead, the same set was sent again with each provider request, including additional requests within a tool loop. Conversation history grew as messages, tool calls, and tool results accumulated. Tool definitions therefore contributed a recurring input cost, while the growing history increased the input required by later requests.

Descriptions are essential. They tell the model when to use a tool, how to resolve IDs, what a result proves, which fields may be changed, and when a write requires confirmation. Removing those guardrails to save tokens can make the agent cheaper and less correct.

The optimization target is precision, not silence:

  • give every tool one unambiguous purpose;

  • remove prose that duplicates another description without changing behavior;

  • keep parameter descriptions focused on constraints the model must know;

  • use server-side conditions and authorization schemes so irrelevant tools are not offered; and

  • consider separate agents or dynamically supplied tools for materially different workflows.

Oracle's APEX 26.1 documentation explicitly supports conditions that control when an on-demand tool is available and lets SQL-query retrieval tools cap their approximate output with Maximum Tokens. Those controls can reduce both unnecessary tool exposure and oversized results.

Be careful with Maximum Tokens: APEX skips an entire row if it exceeds the remaining token allowance, so the tool may return an incomplete set of records. The agent should not assume that the number of returned rows equals the total number of matches. Use a separate count query when an accurate total matters, and make it clear when a tool returns only part of the results.

Two of Genie's 12 available tools were not called anywhere in this suite. Together they represented about 10% of the compact tool JSON. They may be necessary for other production requests, but the benchmark shows the tax paid when a broad agent carries tools that a narrower workflow does not need.

Tool output was the second context multiplier

The 163 tool calls returned 251,908 serialized characters. Two retrieval tools dominated:

Tool Calls Result characters Share of all tool-result characters Largest result
search_tasks 44 151,713 60.2% 12,464
task_details 46 81,849 32.5% 3,202
All other tools 73 18,346 7.3% 714

Search and task details produced 92.7% of all tool-result text. The write tools were tiny; for example, an edit_task_fields result was only 23 characters. The expensive part of an agentic write workflow was therefore not the acknowledgement from the write tool. It was the retrieval and accumulated context required to plan, approve, and verify the change.

Tool output also explains why a more capable model can sometimes reduce tokens. In T03, Sol used five tools and 34,365 input tokens, while Terra used nine tools and 46,492 input tokens. Both passed. Sol reduced input by 26.1%, but its higher rate still made the test cost 42.5% more than Terra.

The lesson is not simply "use fewer tools." A skipped required lookup can cause a quality failure, as Luna demonstrated in T01. The goal is the fewest lookups needed to obtain complete evidence: compact discovery, targeted detail, no redundant retrieval, and a final response grounded in actual tool results.

Models and tools both affected elapsed time

Elapsed time has at least three components in this benchmark:

  1. time inside provider exchanges;

  2. APEX orchestration and tool execution; and

  3. user approval time captured while a run is open.

Terra and Luna each spent about 97 to 98 seconds inside provider exchanges across their complete suites. Sol spent 135.9 seconds, 40.1% more than Terra. Terra’s total recorded run time was 171.8 seconds versus Sol’s 212.8 seconds. That is 19.3% shorter, although these totals include orchestration and approval delays.

Tool use adds latency even when the underlying SQL or PL/SQL is fast because the model must first request the tool and then receive another provider exchange containing the result. Across the suite, one user-visible turn produced an average of 2.84 provider exchanges and 2.36 tool calls. Multiple tool calls can occur in one exchange, so these figures are related but not identical.

Approval workflows need special treatment. The logs show 127.8 seconds outside provider exchanges; they do not isolate how much came from approval waits, tool execution, or other orchestration. For read-only workflows, provider time accounted for most of the elapsed time; for approved writes, the person in the loop can dominate it.

What affected the number of turns?

Nothing in this benchmark changed the planned number of user-visible turns. Every model received exactly 23, because the runbook fixed the dialogue and prohibited coaching or repair messages.

The internal work did vary. Luna used 63 provider exchanges, Terra 68, and Sol 65. Tool calls varied from 50 to 57. The model therefore changed the number of internal tool/model cycles even when the user conversation length stayed fixed.

This distinction matters in production:

  • user turns measure how much back-and-forth a person experiences;

  • provider exchanges determine how often the repeated prompt, tools, and history are billed; and

  • tool calls determine how much external work and result context the agent creates.

The benchmark also understates the cost of weak answers because it did not allow corrective follow-up messages. In a real application, a wrong count, unsupported synthesis, or skipped verification often causes another user turn. The initial Luna answer may cost less, but the completed business outcome may not.

Recommendations for cost-effective APEX agents

Based on this test, I would apply the following design rules:

  1. Evaluate quality before comparing price. A technically complete request is not necessarily a correct result.

  2. Use the cheapest model that clears your real evaluation suite. Base the choice on verified outcomes, not token prices alone.

  3. Measure provider exchanges, not only chat messages. One user turn can contain several billable model requests.

  4. Treat system prompts, tool definitions, and parameter descriptions as recurring context. Keep them precise and make irrelevant tools unavailable when possible.

  5. Design retrieval in layers. Return compact search summaries first, then fetch full records only for selected IDs.

  6. Put limits and pagination in tool contracts. Large search results are replayed into later model requests.

  7. Reset or summarize conversation context at sensible task boundaries. Context reuse is useful, but an ever-growing transcript is an input-token multiplier.

  8. Record correctness alongside telemetry. Tokens, duration, COMPLETE, and missing-result counts are operational signals, not a functional evaluation.

  9. Test reasoning effort separately. Every model in this suite used none, so these results say nothing about the quality, token use, or latency of higher reasoning settings.

Repeated system and tool prefixes also look like good candidates for provider prompt caching. This analysis did not capture or apply cache hits, so that potential saving must be verified against the provider response logs and the actual invoice rather than assumed.

Limitations

These results are specific to one APEX agent, one controlled dataset, one run per model, and one set of prompts. They are useful measurements, not universal model rankings. Network conditions and model behavior can vary. The test did not vary temperature, reasoning effort, tool inventory, prompt wording, page size, or caching independently, so relationships such as context growth and tool-output cost are observational rather than isolated causal experiments.

List-price estimates are not invoices. They omit cache adjustments, any provider-specific accounting not captured by the logger, and taxes. The quality score was intentionally stricter than technical success, but a different application's risk tolerance may weight the failures differently.

Conclusion

The model name is the most visible configuration choice, but it is only one part of an APEX Agent's cost. The model repeatedly consumes the system prompt, every offered tool contract, the active conversation, and the outputs of the tools it chose. Agent design is context design.

All 69 turns completed without a technical failure, yet two scenarios failed their functional checks. That is why cost and correctness need to be measured together: a completed request is not necessarily a completed business outcome.

Before making an agent out of everything, instrument one real agent, build a test suite around actual business work, and measure the whole loop. That is where the useful cost decisions become visible.

📸
The cover image is from Del Mar beach here in San Diego.