Phase 5 โ€” Production Eval

Test 12: Agent Behavior Evaluation

This test is the first agent evaluation exercise in the repo. The goal is not to design a production agent. The goal is to create a small, observable tool-calling system so you can test whether the agent uses tools when needed, avoids tools when not needed, passes the right arguments, and produces a grounded final answer. Every agent step is visible in the generated report.

๐Ÿ“ phase-5-production-eval/agent-evaluation/  |  test12_agent_evaluator.py ยท agent.py ยท tools.py ยท report_generator.py ยท agent_test_cases.xlsx ยท mock_account_data.json

Why Test 12 Exists

Up to Test 11, the system under test was mostly single-step: question in, answer out. Even when RAG was involved, the pipeline was still linear. Agents add a new layer of behavior: they decide whether to call a tool, which tool to call, what arguments to pass, whether to call another tool, and when to stop.

That means evaluation also changes. You can no longer ask only, โ€œWas the final answer correct?โ€ You also have to ask, โ€œDid the system take the right path to get there?โ€

๐Ÿ” The key concept is trajectory evaluation. A trajectory is the sequence of actions the agent took: direct answer, tool call, tool result, second tool call, final answer, or refusal. Test 12 builds the minimum viable agent so that Test 13 can judge those trajectories more formally.

What We Built

The agent folder is self-contained except for the shared model_client.py, which remains common at project root like the other phases.

File Purpose Notes
test12_agent_evaluator.py The script you run. Loads the Excel test cases, runs the agent case by case, compares expected tool path vs actual tool path, prints terminal progress, and generates the HTML trace report. Config-driven via config.yaml
agent.py Minimal agent loop. Forces the model to return JSON actions only: either a tool call or a final answer. Max steps controlled by config / Excel
tools.py Implements the local tools the agent can use. No external services
mock_account_data.json Local structured data for account lookups. Important because the model cannot know this from pretraining
agent_test_cases.xlsx Learner-friendly test case workbook with expected behavior, expected tool path, and why the case exists. Input only
report_generator.py Creates the decorated trace report with visible tool icons and per-case step cards. Output: results/agent_evaluation_report.html

The Tools

Test 12 deliberately keeps the tool set small. This is enough to evaluate agent behavior without turning the exercise into an agent-development project.

๐Ÿ—‚๏ธ
lookup_account_status
Reads exact account data from mock_account_data.json. Best tool for proving the agent used external state.
๐Ÿ“„
search_docs
Searches the local banking documents copied into the Test 12 folder. Used for policy and process questions.
๐Ÿงฎ
calculator
Included as a secondary deterministic tool, but not central to this test run.
โš ๏ธ Why account lookup matters more than calculator. A model can often do arithmetic itself, so tool use may be ambiguous. But the model cannot know the local mock account state unless it calls the lookup tool. That makes tool use visible and testable.

How the Agent Is Constrained

The agent is intentionally strict. It must return one JSON action per step:

{"action":"tool","tool_name":"lookup_account_status","arguments":{"account_id":"A1023"}}
{"action":"final","answer":"..."}

This makes the trajectory inspectable. There is no hidden chain-of-thought, and there are no silent tool invocations. If a tool is used, it appears explicitly in the trace.

๐ŸŽฏ Evaluation-first design. This agent is not optimised for product quality. It is optimised for observability. The trace matters as much as the final answer.

How to Run

cd phase-5-production-eval/agent-evaluation
python3 test12_agent_evaluator.py

The evaluator now shows the total test count before asking whether to run all or a limited subset. It reads its settings from config.yaml:

phase5_agent_evaluation:
  BOT_MODEL: "qwen3.5:9b"
  BOT_PROVIDER: "ollama"
  EXCEL_FILE: "agent_test_cases.xlsx"
  TEST_SHEET: "Test Cases"
  MAX_STEPS: 3

What Was Tested

The workbook contains 15 cases covering five behavior categories:

Actual Run Results

โœ… 14 of 15 cases passed all checks. The latest stable run produced zero malformed actions and zero timeouts. The retry-on-timeout mechanism recovered three cases that stalled under Ollama load. All no-tool, lookup, doc-retrieval, clarification, and refusal cases are fully passing. The single remaining failure is TC10, where the agent stopped after one tool call instead of the expected two-step path.
15
Cases Run
All 15 workbook cases executed end to end with no evaluator errors
14/15
Tool Path Match
Expected tool sequence matched actual sequence in 14 out of 15 cases
14/15
Stop Quality Match
Agent stopped at the right point in 14 cases; TC10 stopped one step too early
0
Malformed Actions
Zero structured action failures after think-tag stripping and retry logic

Per-case summary

Case Question Type Expected Tool Path Actual Tool Path Stop Reason Outcome Notes
TC01 Greeting none none final_answer MATCH Correct direct reply, no unnecessary tool use
TC02 Scope question none none final_answer MATCH Correctly explained capabilities without tools
TC03 Account status lookup_account_status lookup_account_status final_answer MATCH Grounded answer from local structured data
TC04 KYC pending lookup_account_status lookup_account_status final_answer MATCH Correct yes/no answer using lookup result
TC05 Branch lookup lookup_account_status lookup_account_status final_answer MATCH Correct branch returned from mock data
TC06 Dormant reactivation form search_docs search_docs final_answer MATCH Retrieved policy correctly from local docs
TC07 Gold loan eligibility search_docs search_docs final_answer MATCH Used document search rather than model memory
TC08 NextGen account timing search_docs search_docs final_answer MATCH Correct retrieval for operations timing
TC09 Multi-step dormant account lookup_account_statussearch_docs lookup_account_statussearch_docs final_answer MATCH Strong example of correct two-step trajectory with grounded reactivation guidance
TC10 Multi-step issue + next action lookup_account_statussearch_docs lookup_account_status final_answer MISMATCH Agent answered from the lookup result alone without consulting docs for KYC next-action guidance. Stopped one step too early.
TC11 Missing account ID none none final_answer MATCH Correct clarification request; no blind tool call
TC12 Out-of-scope weather none none final_answer MATCH Correct refusal with no irrelevant tool use
TC13 Unknown account ID lookup_account_status lookup_account_status final_answer MATCH Correctly called lookup, received not-found result, and produced a grounded fallback answer
TC14 Explain dormant account search_docs search_docs final_answer MATCH Used docs instead of relying on generic definition
TC15 Generic reactivation guidance search_docs search_docs final_answer MATCH Correct generic guidance from documents

What We Learned

1 โ€” The observable trace idea works

The most important success is not the 93% score. It is that every important action is visible. For every case, you can now inspect whether the agent answered directly, invoked a tool, passed arguments, received a tool result, and then produced a final answer.

2 โ€” Structured lookup is the cleanest proof of tool use

TC03โ€“TC05 show why local structured data is such a good evaluation target. The model cannot know that account A1023 is dormant at branch MG Road unless it calls the lookup tool. That makes correct tool usage obvious and testable.

3 โ€” Thinking models need output stripping before parsing

qwen3.5:9b is a reasoning model. Before producing its answer it emits a <think>...</think> block. The original parser tried to JSON-parse the entire raw output and failed on every case. The fix was a _strip_thinking() helper that removes those blocks before parsing. The same cleaned output is also stored in conversation history so that step-2 prompts are not poisoned by thinking content.

4 โ€” Local model timeouts need retry logic, not just longer timeouts

On an M1 Mac with 16GB RAM, qwen3.5:9b (5.6GB) sometimes exceeds Ollama's 2-minute request timeout under sustained load. Increasing the requests client timeout alone does not help because the Ollama server itself has its own deadline. The solution was a one-retry with 10-second backoff on timeout_error inside agent.py, plus a time.sleep(2) gap between cases to prevent sustained memory pressure. All three timed-out cases in the latest run recovered on retry.

โš ๏ธ TC10 โ€” the one remaining real failure. Account B204 is active with KYC pending. The expected behavior is: lookup the account, then search docs for what the next action for a KYC-pending account should be, then answer. The agent stopped after the lookup and answered from general knowledge instead of consulting the policy documents. The final answer is plausible but not grounded in retrieved docs โ€” exactly the kind of shortcut that makes RAG-based agents unreliable in production.

5 โ€” Multi-step flows need explicit guidance after tool results

To prevent the model from stopping early after a first tool result, the prompt now includes a TOOL RESULT RECEIVED hint that tells the model explicitly to produce a {"action":"final",...} JSON when it already has enough information. Without this, the model would produce flowing prose after the tool result instead of a valid structured action, which was misclassified as malformed_action.

6 โ€” Clarification and refusal are part of correct behavior

TC11 and TC12 are both successful cases even though they do not "complete" the user request directly. Asking for an account ID and refusing an out-of-scope weather question are both correct behaviors. Evaluation must reward those, not penalise them.

What This Sets Up for Test 13

Test 12 is intentionally descriptive. It checks expected tool path vs actual tool path and gives you a visible trace report. Test 13 can now become a true trajectory evaluation test with a judge model such as mistral.

What Test 12 Captures What Test 13 Can Score
Actual tool sequence Was the right tool chosen?
Tool arguments Were the arguments correct and complete?
Tool outputs Did the agent use the tool results correctly?
Stop reason Did it stop in the right place?
Final answer Was the final answer grounded in the trajectory?
๐Ÿ”— Connection to the concepts reference. Test 12 is the first practical use of trajectory evaluation in this repo. It also reinforces earlier ideas: refusal handling, evaluation validity, and disaggregated evaluation. The system can succeed on direct lookup but still fail on multi-step structure. That is exactly why metrics must stay separated.

How to Read the Generated Report

The HTML report at phase-5-production-eval/agent-evaluation/results/agent_evaluation_report.html is the operational output. It uses icons to show tool invocations clearly:

That report is for inspecting the run. This page explains why the test exists and what the run teaches.