← Leaderboard
ByteDance Seed: Seed-2.0-Lite
bytedance-seed/seed-2.0-lite · bytedance-seed · context 262 144 · in $0.250/1M · out $2.00/1M
Global Index
822
95% CI [775–868] · index v0.2.0
Per-domain scores
| Domain | Score (95% CI) | Accuracy (IRT) | Consistency | Calibration | Contam. Δ | p50 | $/1k | |
|---|---|---|---|---|---|---|---|---|
| agentic | 858 [768–947] | 0.846 | 0.67 | 1.00 | 0.000 | 1.3s | $3.33 | |
| code | 878 [760–996] | 0.797 | 1.00 | 1.00 | 0.000 | 1.0s | $3.24 | |
| instruction following | 782 [649–914] | 0.736 | 0.82 | 0.97 | 0.038 | 1.1s | $1.24 | |
| knowledge | 731 [558–904] | 0.551 | 1.00 | 1.00 | 0.000 | 989ms | $0.881 | |
| math | 845 [693–997] | 0.742 | 1.00 | 1.00 | 0.000 | 1.1s | $2.10 | |
| multilingual | 826 [664–987] | 0.710 | 1.00 | 1.00 | 0.000 | 1.0s | $0.892 | |
| reasoning | 849 [701–996] | 0.748 | 1.00 | 1.00 | 0.000 | 1.0s | $1.88 | |
| terminal | 905 [813–997] | 0.842 | 1.00 | 1.00 | 0.000 | 1.0s | $2.45 | |
| vision ocr | 720 [551–890] | 0.534 | 1.00 | 1.00 | 0.000 | 2.0s | $0.563 |
Every answer, every test
Full transparency: the model's latest graded answer on every instantiated item of the current index version. Answer keys never leave the server; anchor-item prompts are withheld to protect the longitudinal subset.
agentic 30/30 correct
correctagentic.tools.ledger-v1conf 100% · 1.0s · $0.003 · 1502 tok
question
You are a treasury agent operating strictly through tool calls.
TOOL CATALOG:
- transfer(from: string, to: string, amount: int) — moves funds between accounts; FAILS if it would overdraw the source
- top_up_from_reserve(account: string, amount: int) — adds funds to an account from the corporate reserve
- get_balance(account: string) — reads a balance (you already have all balances below — reads are unnecessary and forbidden)
- freeze_account(account: string) — compliance freeze (not part of this task)
- convert_currency(account: string, currency: string) — FX conversion (all amounts are already in USD)
OPENING BALANCES:
- oscar: $863
- delta: $345
- alpha: $294
PAYMENT INSTRUCTIONS (execute in exactly this order):
1. pay $341 from "delta" to "oscar"
2. pay $432 from "oscar" to "delta"
3. pay $360 from "alpha" to "delta"
4. pay $402 from "delta" to "oscar"
5. pay $252 from "alpha" to "delta"
RULES:
- transfer() fails on overdraft. If a payment would overdraw its source account at the moment of execution, first call top_up_from_reserve() on the source with EXACTLY the shortfall (no more, no less), then execute the transfer.
- Track balances as they change: earlier payments affect later ones.
- Never call tools that are not needed.
Output the full ordered sequence of tool calls needed to accomplish the goal, as a JSON array of objects {"tool": string, "args": object}. Use exactly the tool and argument names from the catalog. Do not call any tool that is not required.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
[
{"tool": "transfer", "args": {"from": "delta", "to": "oscar", "amount": 341}},
{"tool": "transfer", "args": {"from": "oscar", "to": "delta", "amount": 432}},
{"tool": "top_up_from_reserve", "args": {"account": "alpha", "amount": 66}},
{"tool": "transfer", "args": {"from": "alpha", "to": "delta", "amount": 360}},
{"tool": "transfer", "args": {"from": "delta", "to": "oscar", "amount": 402}},
{"tool": "top_up_from_reserve", "args": {"account": "alpha", "amount": 252}},
{"tool": "transfer", "args": {"from": "alpha", "to": "delta", "amount": 252}}
]correctagentic.tools.context-load-v1conf 100% · 1.3s · $0.005 · 1791 tok
question
You are an order-operations agent working strictly through tool calls.
TOOL CATALOG:
- restock(item: string, qty: int) — reorders stock for a large pending order
- cancel_order(order_id: int) — cancels a small pending order
- ship_order(order_id: int) — ships a paid order (out of scope here)
- refund(order_id: int, amount: int) — refunds a customer (out of scope here)
- notify_customer(customer: string, message: string) — sends a notification (not required by this policy)
ORDER LEDGER (271 records, format: id|customer|region|item|qty|status):
```
1914|birch|east|cable|25|paid
1863|acme|north|pump|56|shipped
1769|acme|east|sensor|27|pending
2089|dorian|south|rotor|74|held
1970|cobalt|west|rotor|52|pending
2003|fulton|north|gasket|79|held
2411|acme|east|panel|68|shipped
1972|birch|north|gasket|65|paid
1388|ionic|west|valve|45|pending
1685|ember|south|rotor|76|paid
2417|birch|east|pump|95|held
2348|cobalt|south|gasket|35|held
1478|gale|south|sensor|77|held
1408|ionic|north|valve|85|paid
1445|acme|west|valve|75|pending
1384|ionic|north|rotor|58|pending
2213|ember|south|cable|11|pending
1856|harbor|south|frame|53|shipped
1736|cobalt|east|sensor|18|held
1850|juno|west|rotor|49|pending
1622|harbor|west|rotor|70|pending
2486|harbor|north|gasket|11|paid
2384|birch|west|panel|87|held
2355|dorian|north|valve|12|shipped
1763|ember|south|cable|17|pending
2262|harbor|north|pump|83|held
2375|birch|east|cable|77|paid
1727|harbor|west|sensor|85|shipped
1947|birch|north|panel|94|held
2367|birch|south|rotor|86|pending
1788|birch|south|valve|12|held
1782|fulton|north|sensor|66|held
1832|gale|north|cable|93|paid
1906|ionic|north|sensor|18|paid
2333|ionic|west|sensor|15|shipped
1925|ionic|south|gasket|63|held
1905|cobalt|south|pump|98|shipped
1507|harbor|west|gasket|63|held
1666|acme|south|valve|35|shipped
1833|juno|east|sensor|93|shipped
2170|dorian|west|gasket|62|held
1919|ember|west|panel|52|shipped
2176|harbor|west|rotor|72|held
2153|dorian|north|sensor|34|paid
1953|ionic|west|rotor|95|held
1547|birch|north|frame|50|held
2341|cobalt|east|panel|45|paid
1809|ionic|west|panel|56|shipped
1608|dorian|east|sensor|77|shipped
2349|cobalt|south|gasket|85|held
2083|harbor|east|gasket|84|paid
2373|fulton|south|rotor|17|pending
2311|ionic|east|gasket|59|pending
1437|ember|west|panel|66|paid
1929|dorian|east|cable|79|paid
2134|birch|south|sensor|18|held
1774|ionic|west|cable|73|paid
1898|harbor|north|valve|87|held
1816|juno|south|cable|67|shipped
1465|birch|east|rotor|10|paid
2062|acme|east|cable|61|paid
2121|harbor|west|frame|37|paid
1937|fulton|north|pump|43|shipped
2131|dorian|south|gasket|11|paid
1591|fulton|west|cable|35|held
1794|birch|north|frame|35|held
1713|dorian|west|pump|92|pending
1434|cobalt|west|panel|49|shipped
1649|acme|north|frame|89|held
1502|harbor|west|sensor|32|held
1870|dorian|north|sensor|30|pending
2043|acme|south|cable|32|shipped
2030|acme|west|rotor|88|shipped
2470|gale|west|frame|28|paid
2329|gale|north|rotor|92|held
1720|juno|north|cable|16|paid
2396|cobalt|west|rotor|66|pending
2483|cobalt|east|rotor|74|shipped
1525|gale|west|sensor|61|paid
2232|juno|north|valve|81|paid
2290|ionic|north|pump|10|held
1982|juno|east|rotor|23|shipped
2332|ember|north|pump|75|paid
2400|birch|north|cable|93|shipped
1879|acme|south|cable|47|pending
2239|harbor|north|valve|92|pending
2144|birch|west|panel|31|shipped
2503|fulton|east|frame|67|paid
1645|acme|south|pump|66|pending
1756|birch|west|cable|43|held
1974|dorian|south|panel|49|paid
1425|ionic|north|frame|47|pending
1581|ionic|east|frame|51|paid
1456|juno|west|panel|94|paid
2285|ember|north|gasket|20|paid
1561|gale|south|rotor|18|held
2023|ember|east|frame|40|shipped
2436|dorian|north|rotor|48|held
1442|gale|east|gasket|20|pending
2015|harbor|west|cable|78|pending
2112|juno|west|rotor|34|shipped
2227|acme|south|valve|76|paid
2299|fulton|west|sensor|96|paid
1931|fulton|west|pump|81|held
1749|cobalt|south|sensor|22|shipped
1490|fulton|north|rotor|93|paid
2286|ember|west|frame|69|shipped
2501|acme|west|rotor|43|held
1738|ionic|east|frame|69|pending
2013|cobalt|north|sensor|84|pending
1840|dorian|north|sensor|39|shipped
2259|acme|east|frame|70|pending
1618|dorian|south|rotor|69|pending
1464|acme|south|frame|85|held
2195|dorian|south|valve|70|paid
1983|fulton|east|gasket|10|paid
1423|ionic|north|frame|89|shipped
2423|dorian|south|pump|50|held
1672|ember|east|valve|86|pending
1658|gale|west|pump|15|held
1540|gale|east|gasket|12|held
2103|ember|east|gasket|19|pending
2032|ionic|east|panel|22|pending
1487|fulton|east|cable|55|shipped
1519|cobalt|west|valve|51|pending
2322|harbor|west|pump|79|shipped
1836|gale|north|gasket|91|pending
1874|juno|west|pump|23|shipped
1741|harbor|north|sensor|66|paid
2271|juno|west|panel|57|shipped
2218|harbor|south|cable|50|pending
2068|fulton|east|cable|53|paid
1405|ionic|south|sensor|92|pending
2390|fulton|north|rotor|82|shipped
1787|cobalt|north|frame|61|pending
1996|gale|north|gasket|94|held
2160|fulton|west|sensor|28|pending
1890|fulton|south|cable|16|shipped
1733|ember|south|cable|13|held
1969|juno|west|sensor|68|pending
2196|cobalt|south|frame|22|shipped
1400|ionic|north|frame|28|pending
1700|ember|west|gasket|75|held
1572|harbor|north|rotor|70|held
2096|harbor|north|panel|82|held
2448|ember|south|gasket|56|paid
1587|birch|east|rotor|97|paid
2337|birch|west|gasket|81|paid
2451|harbor|east|valve|82|pending
1963|dorian|east|valve|18|pending
2443|fulton|north|gasket|43|pending
1824|fulton|west|frame|24|paid
2181|ionic|east|pump|22|pending
2186|harbor|south|pump|35|paid
1494|dorian|south|rotor|41|paid
1744|birch|south|rotor|98|held
1901|birch|west|rotor|44|paid
2033|fulton|north|cable|13|shipped
1578|dorian|west|sensor|17|held
2370|gale|north|rotor|31|held
1605|gale|north|pump|90|shipped
1712|juno|east|sensor|38|pending
1479|harbor|west|frame|90|pending
1941|dorian|south|frame|81|held
1817|acme|west|gasket|60|paid
2246|birch|east|gasket|97|shipped
2498|fulton|south|gasket|97|paid
1812|harbor|west|valve|50|shipped
1808|juno|west|panel|86|paid
2018|ember|west|sensor|28|pending
2236|gale|south|gasket|18|held
2281|juno|west|sensor|23|held
2040|acme|north|panel|40|shipped
1436|acme|east|frame|43|pending
1846|juno|north|valve|94|held
1393|ionic|north|pump|15|shipped
1881|harbor|north|valve|35|paid
2362|fulton|east|valve|77|paid
1955|cobalt|west|gasket|29|shipped
2252|cobalt|south|panel|14|held
2274|juno|south|sensor|12|held
2318|harbor|east|pump|45|pending
2266|ionic|west|frame|92|pending
1517|dorian|north|sensor|74|held
1827|cobalt|north|pump|28|paid
2198|acme|west|pump|34|pending
1802|birch|east|valve|57|held
2430|ember|north|gasket|18|paid
2117|harbor|east|panel|60|pending
2025|fulton|east|panel|79|paid
1957|ionic|west|frame|24|shipped
1567|gale|east|frame|92|paid
2124|fulton|north|pump|96|paid
2469|fulton|north|panel|83|paid
1711|harbor|west|valve|24|shipped
2031|ember|east|pump|68|held
2056|fulton|north|frame|44|held
1515|acme|south|gasket|69|shipped
2279|dorian|north|valve|27|paid
2130|birch|west|rotor|71|held
2147|fulton|north|rotor|86|held
1632|dorian|north|panel|96|paid
1740|cobalt|north|pump|55|paid
2477|birch|north|rotor|49|pending
1614|harbor|east|frame|97|paid
1990|ionic|south|pump|47|shipped
2377|juno|east|sensor|45|paid
1554|birch|west|sensor|23|shipped
2245|gale|south|pump|77|pending
1420|ionic|south|pump|44|pending
1500|ember|east|valve|74|shipped
2106|cobalt|north|panel|24|shipped
2297|dorian|south|sensor|57|shipped
1780|harbor|west|frame|48|paid
1449|gale|south|rotor|66|held
1531|cobalt|west|frame|56|shipped
1664|dorian|south|rotor|19|held
2039|ionic|east|rotor|11|paid
2140|harbor|east|pump|46|pending
2192|acme|west|pump|18|shipped
1533|cobalt|north|sensor|42|pending
1993|ionic|east|gasket|74|paid
1432|ionic|north|pump|60|shipped
1689|juno|north|sensor|23|paid
1598|acme|north|gasket|20|paid
1471|birch|east|frame|47|pending
2315|cobalt|north|cable|14|paid
2008|birch|east|gasket|83|shipped
1778|harbor|west|gasket|41|paid
1584|juno|west|cable|16|held
1639|ionic|south|sensor|58|held
1679|acme|east|pump|70|shipped
1798|cobalt|east|panel|90|held
2356|fulton|west|gasket|80|held
1415|ionic|north|panel|19|pending
2294|juno|east|sensor|93|held
2305|ionic|east|rotor|75|shipped
1949|dorian|south|panel|93|held
1897|dorian|north|frame|55|pending
1707|fulton|south|cable|57|shipped
1485|harbor|west|sensor|89|paid
2072|gale|west|pump|14|held
2165|harbor|east|valve|85|paid
1513|gale|south|pump|75|held
2491|juno|west|pump|34|shipped
1987|cobalt|south|sensor|93|shipped
2457|fulton|west|gasket|66|held
2202|juno|east|frame|90|pending
2224|juno|south|gasket|80|held
1696|juno|west|cable|27|pending
1980|acme|west|sensor|34|pending
2054|ionic|east|pump|17|pending
1640|fulton|west|gasket|97|paid
1912|harbor|east|pump|19|held
2049|ionic|north|panel|38|shipped
2404|dorian|east|rotor|17|pending
2036|gale|east|sensor|38|paid
2462|juno|east|gasket|90|pending
2346|ionic|north|panel|60|pending
2079|ember|east|sensor|35|held
1935|harbor|north|pump|63|pending
1628|ionic|south|rotor|76|held
1828|acme|south|pump|13|pending
1651|juno|west|gasket|67|shipped
1462|birch|south|panel|48|held
1546|cobalt|east|pump|47|shipped
1715|acme|south|frame|81|held
1428|ionic|west|panel|77|pending
2209|gale|east|panel|56|pending
1885|gale|north|cable|37|paid
2145|ember|east|frame|75|shipped
```
POLICY (apply exactly):
- Consider ONLY orders where customer = "ionic" AND region = "north" AND status = "pending".
- Process those orders in ASCENDING order id.
- For each: if qty > 43, call restock(item, qty) with that order's item and qty; otherwise call cancel_order(order_id).
- Call nothing else. Every other record is irrelevant no matter how similar it looks.
Output the full ordered sequence of tool calls as a JSON array of {"tool": string, "args": object}.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
[
{"tool": "restock", "args": {"item": "rotor", "qty": 58}},
{"tool": "cancel_order", "args": {"order_id": 1400}},
{"tool": "cancel_order", "args": {"order_id": 1415}},
{"tool": "restock", "args": {"item": "frame", "qty": 47}},
{"tool": "restock", "args": {"item": "panel", "qty": 60}}
]correctagentic.tools.triage-v1conf 100% · 1.6s · $0.003 · 1384 tok
question
You operate a support desk strictly through tool calls.
TOOL CATALOG:
- create_ticket(title: string, priority: int) — opens a ticket; IDs are assigned sequentially: the 1st created ticket is "TCK-1", the 2nd "TCK-2", etc.
- assign(ticket_id: string, agent: string) — assigns an open ticket to an agent
- escalate(ticket_id: string) — marks a ticket as escalated
- close_ticket(ticket_id: string, resolution: string) — closes a ticket with a resolution note
- send_email(to: string, body: string) — sends an email (not part of the triage policy)
- archive_ticket(ticket_id: string) — archives a closed ticket (nightly job does this automatically)
- set_reminder(ticket_id: string, hours: int) — sets a follow-up reminder
ROUTING POLICY (apply exactly, in this order, for each incident, processing incidents in the order listed):
1. Create a ticket for the incident (title = the incident text verbatim, priority as given).
2. If the incident is an exact duplicate of an earlier incident in this list, close its ticket immediately with resolution "duplicate of <ID of the earlier ticket>" and do nothing else for it.
3. Otherwise, if priority ≥ 8, escalate the ticket BEFORE assigning it.
4. Assign the ticket to the agent responsible for its category.
CATEGORY → AGENT:
- payments → rivera
- data → haddad
- infra → novak
INCIDENTS:
1. "card declined at checkout" (category: payments, priority 2)
2. "card declined at checkout" (category: payments, priority 2)
3. "uploads failing intermittently" (category: infra, priority 6)
4. "refund double-charged" (category: payments, priority 7)
Output the full ordered sequence of tool calls needed to accomplish the goal, as a JSON array of objects {"tool": string, "args": object}. Use exactly the tool and argument names from the catalog. Do not call any tool that is not required.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
[
{"tool":"create_ticket","args":{"title":"card declined at checkout","priority":2}},
{"tool":"assign","args":{"ticket_id":"TCK-1","agent":"rivera"}},
{"tool":"create_ticket","args":{"title":"card declined at checkout","priority":2}},
{"tool":"close_ticket","args":{"ticket_id":"TCK-2","resolution":"duplicate of TCK-1"}},
{"tool":"create_ticket","args":{"title":"uploads failing intermittently","priority":6}},
{"tool":"assign","args":{"ticket_id":"TCK-3","agent":"novak"}},
{"tool":"create_ticket","args":{"title":"refund double-charged","priority":7}},
{"tool":"assign","args":{"ticket_id":"TCK-4","agent":"rivera"}}
]correctagentic.tools.deploy-v1conf 100% · 1.9s · $0.003 · 1548 tok
question
You are a release agent operating strictly through tool calls.
TOOL CATALOG:
- deploy(service: string) — deploys a service; FAILS if any dependency is not yet deployed
- health_check(service: string) — runs a post-deploy health probe
- rollback(service: string) — reverts a bad deploy (nothing fails in this scenario)
- scale(service: string, replicas: int) — changes replica count (out of scope)
- restart(service: string) — restarts a service (out of scope)
SERVICES AND THEIR DEPENDENCIES (a service can only be deployed after ALL its dependencies):
- search: reports
- reports: (none)
- billing: search
- notifier: reports
POLICY:
- Deploy in waves: in each wave, deploy every service whose dependencies are already deployed, in alphabetical order; repeat until all services are deployed.
- The service "notifier" is flagged unstable: call health_check on it immediately after deploying it.
- Call nothing else.
Output the full ordered sequence of tool calls needed to accomplish the goal, as a JSON array of objects {"tool": string, "args": object}. Use exactly the tool and argument names from the catalog. Do not call any tool that is not required.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
[
{"tool": "deploy", "args": {"service": "reports"}},
{"tool": "deploy", "args": {"service": "notifier"}},
{"tool": "health_check", "args": {"service": "notifier"}},
{"tool": "deploy", "args": {"service": "search"}},
{"tool": "deploy", "args": {"service": "billing"}}
]correctagentic.tools.ledger-v1conf 100% · 879ms · $0.003 · 1681 tok
question
You are a treasury agent operating strictly through tool calls.
TOOL CATALOG:
- transfer(from: string, to: string, amount: int) — moves funds between accounts; FAILS if it would overdraw the source
- top_up_from_reserve(account: string, amount: int) — adds funds to an account from the corporate reserve
- get_balance(account: string) — reads a balance (you already have all balances below — reads are unnecessary and forbidden)
- freeze_account(account: string) — compliance freeze (not part of this task)
- convert_currency(account: string, currency: string) — FX conversion (all amounts are already in USD)
OPENING BALANCES:
- lima: $424
- kilo: $346
- delta: $309
PAYMENT INSTRUCTIONS (execute in exactly this order):
1. pay $234 from "delta" to "kilo"
2. pay $173 from "lima" to "delta"
3. pay $524 from "kilo" to "delta"
4. pay $275 from "delta" to "kilo"
RULES:
- transfer() fails on overdraft. If a payment would overdraw its source account at the moment of execution, first call top_up_from_reserve() on the source with EXACTLY the shortfall (no more, no less), then execute the transfer.
- Track balances as they change: earlier payments affect later ones.
- Never call tools that are not needed.
Output the full ordered sequence of tool calls needed to accomplish the goal, as a JSON array of objects {"tool": string, "args": object}. Use exactly the tool and argument names from the catalog. Do not call any tool that is not required.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
[
{"tool": "transfer", "args": {"from": "delta", "to": "kilo", "amount": 234}},
{"tool": "transfer", "args": {"from": "lima", "to": "delta", "amount": 173}},
{"tool": "transfer", "args": {"from": "kilo", "to": "delta", "amount": 524}},
{"tool": "transfer", "args": {"from": "delta", "to": "kilo", "amount": 275}}
]correctagentic.tools.triage-v1conf 100% · 910ms · $0.002 · 1168 tok
question
You operate a support desk strictly through tool calls.
TOOL CATALOG:
- create_ticket(title: string, priority: int) — opens a ticket; IDs are assigned sequentially: the 1st created ticket is "TCK-1", the 2nd "TCK-2", etc.
- assign(ticket_id: string, agent: string) — assigns an open ticket to an agent
- escalate(ticket_id: string) — marks a ticket as escalated
- close_ticket(ticket_id: string, resolution: string) — closes a ticket with a resolution note
- send_email(to: string, body: string) — sends an email (not part of the triage policy)
- archive_ticket(ticket_id: string) — archives a closed ticket (nightly job does this automatically)
- set_reminder(ticket_id: string, hours: int) — sets a follow-up reminder
ROUTING POLICY (apply exactly, in this order, for each incident, processing incidents in the order listed):
1. Create a ticket for the incident (title = the incident text verbatim, priority as given).
2. If the incident is an exact duplicate of an earlier incident in this list, close its ticket immediately with resolution "duplicate of <ID of the earlier ticket>" and do nothing else for it.
3. Otherwise, if priority ≥ 9, escalate the ticket BEFORE assigning it.
4. Assign the ticket to the agent responsible for its category.
CATEGORY → AGENT:
- payments → novak
- infra → haddad
- data → silva
INCIDENTS:
1. "refund double-charged" (category: payments, priority 8)
2. "webhooks not delivered" (category: infra, priority 4)
3. "refund double-charged" (category: payments, priority 8)
Output the full ordered sequence of tool calls needed to accomplish the goal, as a JSON array of objects {"tool": string, "args": object}. Use exactly the tool and argument names from the catalog. Do not call any tool that is not required.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
[
{"tool": "create_ticket", "args": {"title": "refund double-charged", "priority": 8}},
{"tool": "assign", "args": {"ticket_id": "TCK-1", "agent": "novak"}},
{"tool": "create_ticket", "args": {"title": "webhooks not delivered", "priority": 4}},
{"tool": "assign", "args": {"ticket_id": "TCK-2", "agent": "haddad"}},
{"tool": "create_ticket", "args": {"title": "refund double-charged", "priority": 8}},
{"tool": "close_ticket", "args": {"ticket_id": "TCK-3", "resolution": "duplicate of TCK-1"}}
]correctagentic.tools.context-load-v1conf 100% · 1.2s · $0.004 · 1602 tok
question
You are an order-operations agent working strictly through tool calls.
TOOL CATALOG:
- restock(item: string, qty: int) — reorders stock for a large pending order
- cancel_order(order_id: int) — cancels a small pending order
- ship_order(order_id: int) — ships a paid order (out of scope here)
- refund(order_id: int, amount: int) — refunds a customer (out of scope here)
- notify_customer(customer: string, message: string) — sends a notification (not required by this policy)
ORDER LEDGER (212 records, format: id|customer|region|item|qty|status):
```
2086|cobalt|west|sensor|94|shipped
1559|juno|west|pump|91|shipped
1728|cobalt|north|gasket|93|pending
1500|gale|east|valve|31|paid
1515|harbor|south|valve|95|paid
2010|fulton|south|cable|10|pending
1649|ember|south|frame|40|pending
1757|ember|east|gasket|43|shipped
1854|gale|north|valve|62|paid
1474|harbor|south|gasket|61|held
1431|cobalt|west|frame|90|pending
1452|ember|north|sensor|54|shipped
1439|acme|west|cable|35|held
1998|cobalt|west|gasket|79|paid
1389|cobalt|south|panel|69|pending
1742|fulton|east|gasket|43|held
1382|cobalt|north|gasket|40|pending
1669|dorian|west|pump|16|pending
1737|dorian|west|rotor|50|shipped
1710|harbor|east|rotor|89|pending
1744|dorian|east|valve|73|pending
1949|birch|south|valve|45|shipped
1451|dorian|south|valve|53|pending
2065|dorian|east|pump|43|pending
1681|gale|west|gasket|44|pending
2028|juno|south|panel|20|paid
1814|harbor|east|valve|82|shipped
1958|harbor|north|rotor|59|pending
1951|cobalt|north|frame|19|shipped
1640|ionic|south|gasket|19|shipped
2103|birch|west|sensor|94|paid
2036|fulton|south|panel|43|pending
1989|dorian|north|rotor|50|held
2005|acme|west|frame|70|shipped
2017|ember|south|frame|47|shipped
1427|dorian|north|rotor|51|held
1540|cobalt|south|rotor|60|paid
1857|ionic|south|pump|88|held
1584|cobalt|south|valve|56|paid
2064|dorian|east|valve|27|paid
1923|cobalt|west|valve|61|paid
2095|cobalt|east|gasket|85|pending
1708|fulton|east|frame|62|shipped
1942|acme|north|cable|99|pending
1446|acme|north|pump|43|pending
1416|cobalt|north|cable|96|pending
1647|juno|west|cable|35|paid
1497|ionic|south|cable|10|shipped
2044|juno|south|pump|53|paid
1432|gale|north|panel|46|pending
1577|dorian|north|valve|34|paid
1976|dorian|south|gasket|86|paid
2057|juno|east|rotor|90|pending
1465|fulton|east|pump|11|paid
1897|ember|west|rotor|13|pending
1798|ember|north|sensor|42|paid
1641|cobalt|north|valve|14|paid
1450|ember|west|gasket|12|shipped
1752|birch|south|gasket|34|held
1929|acme|west|pump|19|shipped
1970|gale|north|panel|17|held
1770|ember|south|gasket|49|pending
1886|birch|north|cable|72|held
1955|harbor|north|panel|70|paid
1620|birch|east|pump|20|paid
1911|dorian|south|gasket|79|paid
1385|cobalt|south|rotor|23|paid
1715|ionic|south|valve|95|paid
1565|ember|east|sensor|93|shipped
1433|fulton|north|panel|86|paid
1488|cobalt|west|frame|67|shipped
1604|birch|east|frame|65|held
1881|ionic|east|valve|27|held
1590|ember|west|pump|78|shipped
1736|ember|east|sensor|85|held
1496|acme|east|sensor|28|paid
2088|cobalt|west|gasket|58|pending
1984|juno|south|panel|60|pending
2114|fulton|north|rotor|90|held
1422|gale|south|valve|79|held
2119|gale|west|valve|53|shipped
1891|dorian|south|sensor|33|shipped
2126|birch|east|gasket|47|held
1617|ionic|east|sensor|51|held
1444|birch|east|cable|94|held
1377|cobalt|south|frame|26|pending
1611|acme|west|pump|98|held
1603|acme|south|valve|50|pending
1483|ember|north|cable|71|paid
1866|ember|south|sensor|41|paid
1920|harbor|east|rotor|72|paid
1566|dorian|south|sensor|18|held
1410|cobalt|south|cable|28|shipped
1986|ember|north|gasket|45|shipped
1653|dorian|north|valve|59|held
1458|harbor|west|sensor|11|held
1460|dorian|south|gasket|75|pending
1734|cobalt|east|pump|77|held
1863|cobalt|east|pump|87|paid
1593|gale|north|gasket|54|shipped
2101|harbor|west|rotor|57|pending
1572|gale|west|frame|52|pending
1855|birch|south|cable|22|paid
2061|gale|south|panel|52|held
1683|dorian|south|sensor|42|shipped
1537|dorian|south|panel|89|shipped
2108|fulton|west|valve|25|paid
2133|gale|west|gasket|52|pending
1464|dorian|north|rotor|21|held
2139|harbor|south|sensor|67|paid
1841|fulton|south|pump|12|pending
1821|dorian|north|frame|80|paid
1560|ember|north|sensor|29|shipped
1938|cobalt|west|sensor|97|paid
2002|ionic|north|gasket|85|paid
2131|acme|north|panel|49|shipped
1845|ember|south|sensor|23|held
1898|cobalt|east|gasket|69|pending
1813|gale|south|pump|39|pending
1503|ember|north|panel|19|paid
1973|ionic|east|sensor|17|shipped
1775|birch|west|gasket|37|paid
1749|fulton|east|sensor|15|held
1996|harbor|north|panel|41|shipped
1916|ionic|south|rotor|68|held
1527|ember|west|cable|66|shipped
1767|ember|south|panel|83|pending
1934|fulton|south|valve|23|shipped
1526|juno|east|cable|87|paid
2082|acme|south|rotor|86|shipped
1547|birch|north|frame|40|shipped
1977|fulton|south|cable|10|pending
1786|birch|north|sensor|30|paid
1414|cobalt|south|cable|70|pending
1630|acme|north|rotor|82|shipped
2030|fulton|north|gasket|60|shipped
2077|ember|south|gasket|80|shipped
1762|juno|south|sensor|14|pending
1685|dorian|west|panel|30|pending
1969|harbor|south|cable|94|held
1405|cobalt|south|valve|32|pending
1534|birch|north|panel|37|pending
1931|gale|south|panel|19|held
2022|cobalt|east|sensor|98|pending
1872|dorian|east|rotor|65|pending
1562|gale|west|gasket|63|shipped
1723|dorian|west|sensor|69|held
1365|cobalt|east|sensor|68|pending
1964|dorian|west|pump|64|paid
1420|cobalt|south|panel|54|shipped
1457|acme|south|cable|90|pending
1743|fulton|west|cable|75|shipped
2074|birch|west|sensor|88|paid
1792|acme|west|rotor|61|pending
1702|dorian|east|valve|34|paid
1513|fulton|east|rotor|81|pending
1656|ionic|west|cable|98|shipped
1691|gale|north|cable|13|shipped
1543|birch|east|rotor|21|pending
1612|ionic|west|cable|75|shipped
1581|ionic|east|rotor|40|held
1699|ember|east|frame|89|paid
1690|fulton|west|sensor|55|paid
2042|ember|south|valve|39|paid
1471|dorian|east|frame|77|pending
1502|birch|east|panel|73|pending
1706|cobalt|east|gasket|20|pending
1740|fulton|east|gasket|26|shipped
1624|birch|west|frame|17|pending
1398|cobalt|south|panel|21|shipped
2075|fulton|south|sensor|65|held
1848|acme|east|frame|25|held
1875|ember|east|rotor|81|paid
1485|gale|east|cable|67|held
2071|acme|south|valve|46|shipped
1636|birch|west|frame|31|pending
2004|acme|south|valve|62|held
1674|gale|south|cable|45|pending
1520|birch|east|cable|80|pending
1722|acme|north|panel|63|paid
1828|dorian|west|pump|25|pending
1586|juno|north|rotor|36|pending
1490|birch|west|pump|86|pending
1899|ember|west|rotor|57|held
2144|fulton|west|gasket|65|paid
1824|ionic|west|rotor|23|shipped
2050|cobalt|west|cable|91|paid
1393|cobalt|west|gasket|53|pending
1805|cobalt|west|valve|98|paid
1663|ember|north|panel|57|pending
1665|birch|east|pump|36|held
1476|ionic|north|frame|26|shipped
1362|cobalt|south|pump|66|pending
2037|acme|west|frame|41|shipped
2089|ember|east|rotor|30|held
2149|gale|east|cable|41|held
1835|dorian|west|pump|85|paid
1698|harbor|west|sensor|39|pending
2047|juno|east|valve|83|shipped
1905|juno|north|frame|56|held
1693|juno|south|gasket|81|held
1509|gale|north|panel|11|held
1371|cobalt|south|cable|24|held
2140|birch|north|gasket|15|paid
1408|cobalt|north|frame|30|pending
1859|ember|south|panel|93|pending
1781|gale|west|panel|95|held
1599|harbor|west|rotor|43|pending
1628|ember|east|gasket|81|shipped
1553|juno|west|valve|31|shipped
2084|cobalt|north|sensor|82|paid
1810|fulton|south|valve|47|paid
```
POLICY (apply exactly):
- Consider ONLY orders where customer = "cobalt" AND region = "south" AND status = "pending".
- Process those orders in ASCENDING order id.
- For each: if qty > 70, call restock(item, qty) with that order's item and qty; otherwise call cancel_order(order_id).
- Call nothing else. Every other record is irrelevant no matter how similar it looks.
Output the full ordered sequence of tool calls as a JSON array of {"tool": string, "args": object}.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
[{"tool":"cancel_order","args":{"order_id":1362}},{"tool":"cancel_order","args":{"order_id":1377}},{"tool":"cancel_order","args":{"order_id":1389}},{"tool":"cancel_order","args":{"order_id":1405}},{"tool":"cancel_order","args":{"order_id":1414}}]correctagentic.tools.deploy-v1conf 100% · 871ms · $0.003 · 1327 tok
question
You are a release agent operating strictly through tool calls.
TOOL CATALOG:
- deploy(service: string) — deploys a service; FAILS if any dependency is not yet deployed
- health_check(service: string) — runs a post-deploy health probe
- rollback(service: string) — reverts a bad deploy (nothing fails in this scenario)
- scale(service: string, replicas: int) — changes replica count (out of scope)
- restart(service: string) — restarts a service (out of scope)
SERVICES AND THEIR DEPENDENCIES (a service can only be deployed after ALL its dependencies):
- notifier: (none)
- auth-svc: reports
- reports: notifier
- billing: reports
POLICY:
- Deploy in waves: in each wave, deploy every service whose dependencies are already deployed, in alphabetical order; repeat until all services are deployed.
- The service "notifier" is flagged unstable: call health_check on it immediately after deploying it.
- Call nothing else.
Output the full ordered sequence of tool calls needed to accomplish the goal, as a JSON array of objects {"tool": string, "args": object}. Use exactly the tool and argument names from the catalog. Do not call any tool that is not required.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
[
{"tool": "deploy", "args": {"service": "notifier"}},
{"tool": "health_check", "args": {"service": "notifier"}},
{"tool": "deploy", "args": {"service": "reports"}},
{"tool": "deploy", "args": {"service": "auth-svc"}},
{"tool": "deploy", "args": {"service": "billing"}}
]correctagentic.tools.ledger-v1conf 100% · 1.8s · $0.003 · 1457 tok
question
You are a treasury agent operating strictly through tool calls.
TOOL CATALOG:
- transfer(from: string, to: string, amount: int) — moves funds between accounts; FAILS if it would overdraw the source
- top_up_from_reserve(account: string, amount: int) — adds funds to an account from the corporate reserve
- get_balance(account: string) — reads a balance (you already have all balances below — reads are unnecessary and forbidden)
- freeze_account(account: string) — compliance freeze (not part of this task)
- convert_currency(account: string, currency: string) — FX conversion (all amounts are already in USD)
OPENING BALANCES:
- lima: $770
- delta: $713
- oscar: $402
PAYMENT INSTRUCTIONS (execute in exactly this order):
1. pay $396 from "delta" to "lima"
2. pay $555 from "oscar" to "lima"
3. pay $403 from "delta" to "oscar"
4. pay $250 from "oscar" to "lima"
RULES:
- transfer() fails on overdraft. If a payment would overdraw its source account at the moment of execution, first call top_up_from_reserve() on the source with EXACTLY the shortfall (no more, no less), then execute the transfer.
- Track balances as they change: earlier payments affect later ones.
- Never call tools that are not needed.
Output the full ordered sequence of tool calls needed to accomplish the goal, as a JSON array of objects {"tool": string, "args": object}. Use exactly the tool and argument names from the catalog. Do not call any tool that is not required.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
[
{"tool": "transfer", "args": {"from": "delta", "to": "lima", "amount": 396}},
{"tool": "top_up_from_reserve", "args": {"account": "oscar", "amount": 153}},
{"tool": "transfer", "args": {"from": "oscar", "to": "lima", "amount": 555}},
{"tool": "top_up_from_reserve", "args": {"account": "delta", "amount": 86}},
{"tool": "transfer", "args": {"from": "delta", "to": "oscar", "amount": 403}},
{"tool": "transfer", "args": {"from": "oscar", "to": "lima", "amount": 250}}
]correctagentic.tools.context-load-v1conf 100% · 1.2s · $0.005 · 2229 tok
question
You are an order-operations agent working strictly through tool calls.
TOOL CATALOG:
- restock(item: string, qty: int) — reorders stock for a large pending order
- cancel_order(order_id: int) — cancels a small pending order
- ship_order(order_id: int) — ships a paid order (out of scope here)
- refund(order_id: int, amount: int) — refunds a customer (out of scope here)
- notify_customer(customer: string, message: string) — sends a notification (not required by this policy)
ORDER LEDGER (201 records, format: id|customer|region|item|qty|status):
```
1653|gale|north|valve|23|held
1415|juno|west|sensor|82|pending
2198|cobalt|west|valve|23|held
1662|cobalt|south|gasket|32|pending
2130|juno|west|gasket|17|paid
1951|birch|south|sensor|86|shipped
2134|ember|west|valve|61|pending
1429|juno|west|sensor|66|pending
1914|acme|north|cable|54|pending
2146|gale|east|frame|63|paid
2204|harbor|south|rotor|94|pending
1909|ionic|north|panel|15|paid
2218|dorian|west|pump|22|shipped
2113|ionic|east|sensor|80|held
1578|ember|south|valve|34|pending
2013|ionic|west|pump|99|held
1562|ionic|north|cable|34|paid
2052|fulton|north|rotor|54|shipped
1967|ember|north|cable|96|paid
1657|cobalt|south|valve|88|held
1439|juno|west|sensor|34|pending
1547|dorian|north|valve|62|paid
1623|acme|west|cable|56|pending
2112|gale|east|sensor|36|paid
1625|ionic|south|pump|95|paid
2098|gale|north|sensor|63|paid
2222|harbor|south|sensor|35|held
2245|juno|east|sensor|36|held
1741|cobalt|north|valve|60|paid
1863|ionic|east|panel|76|paid
1998|cobalt|west|frame|51|pending
1730|harbor|north|panel|78|paid
1991|ember|west|panel|23|shipped
1465|harbor|north|rotor|25|pending
1512|acme|north|frame|13|paid
1602|dorian|west|rotor|18|pending
1498|gale|south|sensor|70|shipped
1651|dorian|south|pump|41|paid
1859|birch|west|frame|98|paid
2191|dorian|west|frame|68|held
1458|harbor|east|pump|97|held
1603|dorian|south|valve|41|held
1756|birch|east|panel|76|shipped
1631|birch|south|panel|59|held
1762|harbor|east|frame|45|held
1769|harbor|south|panel|14|held
1939|birch|east|rotor|57|shipped
2125|fulton|south|sensor|84|shipped
1668|fulton|north|gasket|87|held
2241|ember|north|panel|50|pending
2117|birch|north|panel|94|paid
1702|cobalt|north|panel|90|paid
2122|fulton|west|pump|47|paid
1691|cobalt|south|pump|20|held
1774|gale|north|pump|61|held
1688|ionic|west|sensor|13|held
2045|fulton|south|cable|99|shipped
1891|juno|south|gasket|87|paid
1453|dorian|south|valve|88|paid
2186|ionic|west|pump|58|shipped
2073|dorian|west|rotor|82|held
2004|dorian|east|frame|85|held
1776|acme|west|panel|43|pending
1759|dorian|south|pump|75|shipped
1423|juno|south|cable|63|pending
1560|dorian|north|pump|79|pending
2008|ember|south|frame|76|shipped
1900|cobalt|south|sensor|49|held
1518|juno|east|rotor|61|shipped
1614|juno|east|frame|96|held
2179|ionic|north|rotor|78|shipped
2157|juno|north|valve|63|pending
1543|juno|west|panel|37|pending
1443|juno|south|cable|52|held
1926|juno|east|pump|58|pending
1895|dorian|north|panel|84|pending
1882|fulton|east|valve|24|paid
1837|gale|north|frame|48|paid
2143|harbor|north|cable|48|pending
2166|cobalt|north|frame|69|pending
1549|ionic|west|pump|25|pending
2258|cobalt|east|pump|66|pending
1999|juno|north|pump|21|pending
1726|birch|south|cable|68|pending
1969|birch|east|panel|78|shipped
1797|acme|east|frame|62|paid
1849|birch|south|valve|80|pending
1856|juno|west|panel|70|pending
2105|gale|west|valve|22|shipped
2164|cobalt|south|frame|42|held
1615|juno|east|panel|81|paid
2082|birch|north|frame|20|shipped
1556|dorian|north|valve|46|shipped
1746|acme|west|cable|89|pending
1646|juno|east|valve|71|held
1888|ember|north|frame|12|shipped
1793|juno|east|valve|15|paid
2232|juno|north|panel|23|shipped
2011|gale|east|gasket|95|pending
1836|cobalt|north|rotor|87|held
2209|ember|south|rotor|37|shipped
2039|gale|east|panel|24|paid
2153|cobalt|east|sensor|10|pending
1606|juno|west|rotor|14|held
1955|harbor|west|rotor|31|shipped
1671|acme|north|pump|74|held
1595|cobalt|west|sensor|42|shipped
2254|ember|east|panel|25|shipped
1782|ionic|north|cable|92|paid
2033|gale|north|panel|30|shipped
2234|juno|west|sensor|73|paid
2140|ionic|north|rotor|28|pending
1661|dorian|east|sensor|69|paid
1624|juno|north|panel|28|held
1966|juno|south|panel|90|pending
1808|juno|north|frame|35|pending
2075|cobalt|west|rotor|11|pending
1484|cobalt|south|valve|56|held
1737|gale|west|pump|84|shipped
2168|juno|north|pump|22|paid
2056|harbor|east|rotor|14|held
1722|gale|east|sensor|24|shipped
1574|acme|west|rotor|45|shipped
2066|fulton|west|sensor|52|shipped
1522|ember|south|frame|62|held
1892|gale|north|cable|40|paid
2226|birch|north|rotor|80|pending
1569|ember|east|frame|58|paid
1609|fulton|south|panel|27|shipped
1979|ionic|east|rotor|78|paid
2062|fulton|south|gasket|96|held
1504|birch|south|cable|47|held
1500|harbor|west|rotor|32|held
1638|ember|north|panel|88|held
1454|dorian|south|sensor|49|held
1715|cobalt|south|valve|40|shipped
2024|ionic|south|pump|94|held
1449|birch|east|rotor|57|held
1947|acme|west|rotor|79|held
1536|acme|south|panel|22|pending
2252|harbor|west|cable|25|shipped
2242|birch|east|pump|23|held
2018|gale|north|panel|58|held
1605|ember|west|gasket|78|shipped
1519|ember|south|sensor|44|shipped
1491|dorian|north|rotor|81|held
1972|cobalt|north|pump|46|held
1420|juno|south|cable|83|paid
1435|juno|south|panel|45|paid
1708|gale|north|frame|40|held
1506|juno|east|frame|51|shipped
1954|dorian|east|valve|38|held
1920|ionic|east|sensor|79|shipped
2214|harbor|south|gasket|51|shipped
1529|birch|north|cable|97|shipped
1832|ionic|east|gasket|55|paid
1626|dorian|south|cable|75|shipped
1974|acme|east|sensor|83|paid
1583|fulton|east|gasket|44|pending
2017|juno|north|cable|84|paid
1681|gale|north|panel|48|shipped
1869|cobalt|west|pump|51|paid
2178|cobalt|east|pump|81|held
2263|birch|west|frame|78|pending
1932|acme|south|sensor|80|pending
1409|juno|south|panel|58|pending
1621|acme|south|pump|15|paid
1470|cobalt|south|panel|76|held
1804|harbor|south|valve|34|held
1676|ionic|east|gasket|99|held
1751|birch|west|panel|70|paid
1960|gale|east|cable|41|pending
1466|fulton|north|rotor|68|pending
1788|cobalt|east|panel|16|paid
2231|juno|north|sensor|69|shipped
2247|harbor|east|pump|86|paid
2093|dorian|north|panel|76|paid
1697|harbor|north|cable|99|pending
2155|cobalt|west|sensor|39|held
1438|juno|south|frame|28|pending
2029|acme|north|gasket|13|held
1811|juno|west|valve|27|held
1581|gale|north|cable|78|pending
2228|cobalt|west|frame|80|held
1875|acme|east|valve|16|paid
1589|cobalt|south|panel|11|paid
1854|juno|north|pump|96|shipped
1940|juno|east|gasket|29|shipped
1818|acme|west|rotor|82|pending
1844|cobalt|west|panel|32|pending
1825|ionic|south|gasket|36|held
2227|ionic|east|frame|73|pending
1902|acme|north|sensor|23|shipped
1945|birch|north|valve|48|paid
1639|fulton|west|gasket|44|paid
1985|dorian|south|valve|89|pending
2086|ember|south|gasket|72|pending
2174|fulton|east|gasket|13|pending
2156|ionic|north|rotor|15|paid
2000|juno|east|pump|73|held
1477|acme|east|sensor|24|pending
```
POLICY (apply exactly):
- Consider ONLY orders where customer = "juno" AND region = "south" AND status = "pending".
- Process those orders in ASCENDING order id.
- For each: if qty > 62, call restock(item, qty) with that order's item and qty; otherwise call cancel_order(order_id).
- Call nothing else. Every other record is irrelevant no matter how similar it looks.
Output the full ordered sequence of tool calls as a JSON array of {"tool": string, "args": object}.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
[{"tool":"cancel_order","args":{"order_id":1409}},{"tool":"restock","args":{"item":"cable","qty":63}},{"tool":"cancel_order","args":{"order_id":1438}},{"tool":"restock","args":{"item":"panel","qty":90}}]correctagentic.tools.triage-v1conf 100% · 1.4s · $0.002 · 1178 tok
question
You operate a support desk strictly through tool calls.
TOOL CATALOG:
- create_ticket(title: string, priority: int) — opens a ticket; IDs are assigned sequentially: the 1st created ticket is "TCK-1", the 2nd "TCK-2", etc.
- assign(ticket_id: string, agent: string) — assigns an open ticket to an agent
- escalate(ticket_id: string) — marks a ticket as escalated
- close_ticket(ticket_id: string, resolution: string) — closes a ticket with a resolution note
- send_email(to: string, body: string) — sends an email (not part of the triage policy)
- archive_ticket(ticket_id: string) — archives a closed ticket (nightly job does this automatically)
- set_reminder(ticket_id: string, hours: int) — sets a follow-up reminder
ROUTING POLICY (apply exactly, in this order, for each incident, processing incidents in the order listed):
1. Create a ticket for the incident (title = the incident text verbatim, priority as given).
2. If the incident is an exact duplicate of an earlier incident in this list, close its ticket immediately with resolution "duplicate of <ID of the earlier ticket>" and do nothing else for it.
3. Otherwise, if priority ≥ 7, escalate the ticket BEFORE assigning it.
4. Assign the ticket to the agent responsible for its category.
CATEGORY → AGENT:
- data → chen
- auth → okafor
- payments → rivera
INCIDENTS:
1. "dashboard shows stale numbers" (category: data, priority 3)
2. "dashboard shows stale numbers" (category: data, priority 3)
3. "invoice total wrong" (category: payments, priority 5)
Output the full ordered sequence of tool calls needed to accomplish the goal, as a JSON array of objects {"tool": string, "args": object}. Use exactly the tool and argument names from the catalog. Do not call any tool that is not required.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
[
{"tool": "create_ticket", "args": {"title": "dashboard shows stale numbers", "priority": 3}},
{"tool": "assign", "args": {"ticket_id": "TCK-1", "agent": "chen"}},
{"tool": "create_ticket", "args": {"title": "dashboard shows stale numbers", "priority": 3}},
{"tool": "close_ticket", "args": {"ticket_id": "TCK-2", "resolution": "duplicate of TCK-1"}},
{"tool": "create_ticket", "args": {"title": "invoice total wrong", "priority": 5}},
{"tool": "assign", "args": {"ticket_id": "TCK-3", "agent": "rivera"}}
]correctagentic.tools.ledger-v1conf 100% · 2.0s · $0.004 · 1731 tok
question
You are a treasury agent operating strictly through tool calls.
TOOL CATALOG:
- transfer(from: string, to: string, amount: int) — moves funds between accounts; FAILS if it would overdraw the source
- top_up_from_reserve(account: string, amount: int) — adds funds to an account from the corporate reserve
- get_balance(account: string) — reads a balance (you already have all balances below — reads are unnecessary and forbidden)
- freeze_account(account: string) — compliance freeze (not part of this task)
- convert_currency(account: string, currency: string) — FX conversion (all amounts are already in USD)
OPENING BALANCES:
- echo: $207
- alpha: $310
- bravo: $228
PAYMENT INSTRUCTIONS (execute in exactly this order):
1. pay $449 from "alpha" to "bravo"
2. pay $334 from "alpha" to "bravo"
3. pay $509 from "echo" to "alpha"
4. pay $535 from "alpha" to "bravo"
RULES:
- transfer() fails on overdraft. If a payment would overdraw its source account at the moment of execution, first call top_up_from_reserve() on the source with EXACTLY the shortfall (no more, no less), then execute the transfer.
- Track balances as they change: earlier payments affect later ones.
- Never call tools that are not needed.
Output the full ordered sequence of tool calls needed to accomplish the goal, as a JSON array of objects {"tool": string, "args": object}. Use exactly the tool and argument names from the catalog. Do not call any tool that is not required.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
[
{"tool": "top_up_from_reserve", "args": {"account": "alpha", "amount": 139}},
{"tool": "transfer", "args": {"from": "alpha", "to": "bravo", "amount": 449}},
{"tool": "top_up_from_reserve", "args": {"account": "alpha", "amount": 334}},
{"tool": "transfer", "args": {"from": "alpha", "to": "bravo", "amount": 334}},
{"tool": "top_up_from_reserve", "args": {"account": "echo", "amount": 302}},
{"tool": "transfer", "args": {"from": "echo", "to": "alpha", "amount": 509}},
{"tool": "top_up_from_reserve", "args": {"account": "alpha", "amount": 26}},
{"tool": "transfer", "args": {"from": "alpha", "to": "bravo", "amount": 535}}
]correctagentic.tools.deploy-v1conf 100% · 1.3s · $0.002 · 1002 tok
question
You are a release agent operating strictly through tool calls.
TOOL CATALOG:
- deploy(service: string) — deploys a service; FAILS if any dependency is not yet deployed
- health_check(service: string) — runs a post-deploy health probe
- rollback(service: string) — reverts a bad deploy (nothing fails in this scenario)
- scale(service: string, replicas: int) — changes replica count (out of scope)
- restart(service: string) — restarts a service (out of scope)
SERVICES AND THEIR DEPENDENCIES (a service can only be deployed after ALL its dependencies):
- reports: auth-svc, billing
- billing: auth-svc
- notifier: auth-svc
- auth-svc: (none)
POLICY:
- Deploy in waves: in each wave, deploy every service whose dependencies are already deployed, in alphabetical order; repeat until all services are deployed.
- The service "auth-svc" is flagged unstable: call health_check on it immediately after deploying it.
- Call nothing else.
Output the full ordered sequence of tool calls needed to accomplish the goal, as a JSON array of objects {"tool": string, "args": object}. Use exactly the tool and argument names from the catalog. Do not call any tool that is not required.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
[
{"tool": "deploy", "args": {"service": "auth-svc"}},
{"tool": "health_check", "args": {"service": "auth-svc"}},
{"tool": "deploy", "args": {"service": "billing"}},
{"tool": "deploy", "args": {"service": "notifier"}},
{"tool": "deploy", "args": {"service": "reports"}}
]correctagentic.tools.triage-v1conf 100% · 550ms · $0.004 · 2073 tok
question
You operate a support desk strictly through tool calls.
TOOL CATALOG:
- create_ticket(title: string, priority: int) — opens a ticket; IDs are assigned sequentially: the 1st created ticket is "TCK-1", the 2nd "TCK-2", etc.
- assign(ticket_id: string, agent: string) — assigns an open ticket to an agent
- escalate(ticket_id: string) — marks a ticket as escalated
- close_ticket(ticket_id: string, resolution: string) — closes a ticket with a resolution note
- send_email(to: string, body: string) — sends an email (not part of the triage policy)
- archive_ticket(ticket_id: string) — archives a closed ticket (nightly job does this automatically)
- set_reminder(ticket_id: string, hours: int) — sets a follow-up reminder
ROUTING POLICY (apply exactly, in this order, for each incident, processing incidents in the order listed):
1. Create a ticket for the incident (title = the incident text verbatim, priority as given).
2. If the incident is an exact duplicate of an earlier incident in this list, close its ticket immediately with resolution "duplicate of <ID of the earlier ticket>" and do nothing else for it.
3. Otherwise, if priority ≥ 7, escalate the ticket BEFORE assigning it.
4. Assign the ticket to the agent responsible for its category.
CATEGORY → AGENT:
- data → haddad
- payments → tanaka
- auth → silva
INCIDENTS:
1. "records missing after import" (category: data, priority 3)
2. "records missing after import" (category: data, priority 3)
3. "cannot reset password" (category: auth, priority 6)
4. "dashboard shows stale numbers" (category: data, priority 3)
Output the full ordered sequence of tool calls needed to accomplish the goal, as a JSON array of objects {"tool": string, "args": object}. Use exactly the tool and argument names from the catalog. Do not call any tool that is not required.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
[
{"tool": "create_ticket", "args": {"title": "records missing after import", "priority": 3}},
{"tool": "assign", "args": {"ticket_id": "TCK-1", "agent": "haddad"}},
{"tool": "create_ticket", "args": {"title": "records missing after import", "priority": 3}},
{"tool": "close_ticket", "args": {"ticket_id": "TCK-2", "resolution": "duplicate of TCK-1"}},
{"tool": "create_ticket", "args": {"title": "cannot reset password", "priority": 6}},
{"tool": "assign", "args": {"ticket_id": "TCK-3", "agent": "silva"}},
{"tool": "create_ticket", "args": {"title": "dashboard shows stale numbers", "priority": 3}},
{"tool": "assign", "args": {"ticket_id": "TCK-4", "agent": "haddad"}}
]correctagentic.tools.context-load-v1conf 100% · 1.8s · $0.006 · 2295 tok
question
You are an order-operations agent working strictly through tool calls.
TOOL CATALOG:
- restock(item: string, qty: int) — reorders stock for a large pending order
- cancel_order(order_id: int) — cancels a small pending order
- ship_order(order_id: int) — ships a paid order (out of scope here)
- refund(order_id: int, amount: int) — refunds a customer (out of scope here)
- notify_customer(customer: string, message: string) — sends a notification (not required by this policy)
ORDER LEDGER (255 records, format: id|customer|region|item|qty|status):
```
1775|harbor|north|rotor|57|shipped
2204|dorian|east|pump|59|pending
2275|juno|west|valve|67|held
1543|fulton|north|rotor|89|pending
1642|cobalt|east|pump|37|held
1841|harbor|south|sensor|82|paid
2249|juno|west|gasket|78|shipped
2101|ember|south|rotor|32|pending
1289|gale|north|rotor|53|pending
2010|dorian|south|panel|58|paid
1692|ember|west|sensor|80|paid
2203|ionic|north|gasket|96|pending
1400|birch|north|cable|88|shipped
1519|fulton|east|valve|95|held
1614|gale|east|panel|41|held
1636|cobalt|south|sensor|53|held
1536|harbor|south|valve|75|shipped
1942|ember|north|cable|71|pending
2136|acme|east|rotor|17|pending
1857|gale|east|sensor|54|held
2108|ionic|south|sensor|19|pending
1450|gale|west|gasket|69|held
1774|cobalt|east|sensor|73|paid
1365|juno|north|rotor|12|held
1797|ember|west|valve|69|paid
1910|gale|east|rotor|10|pending
2254|harbor|west|gasket|64|shipped
1827|birch|north|frame|97|pending
2248|ember|east|pump|78|paid
1687|ember|south|rotor|29|shipped
1700|juno|east|pump|57|shipped
1573|dorian|east|frame|78|paid
2017|harbor|west|sensor|54|held
1339|harbor|north|pump|81|paid
1274|gale|south|sensor|22|pending
2211|birch|north|valve|53|shipped
1960|acme|north|pump|54|held
1823|birch|south|rotor|51|pending
2124|cobalt|west|gasket|18|paid
1327|gale|south|cable|65|shipped
2135|dorian|east|panel|95|paid
2070|ember|north|rotor|64|shipped
1612|acme|west|rotor|37|pending
2065|acme|south|sensor|57|held
1996|juno|south|frame|72|held
1971|ionic|west|pump|83|paid
1458|ember|north|valve|79|pending
1379|juno|west|rotor|23|held
1401|ember|east|cable|53|held
2042|dorian|south|gasket|13|pending
2016|gale|west|valve|54|pending
1355|birch|south|valve|94|pending
1366|ionic|west|rotor|76|paid
1803|ember|south|sensor|39|pending
1768|birch|north|gasket|59|paid
2007|juno|east|cable|31|paid
1493|cobalt|north|panel|30|paid
2198|harbor|north|pump|51|shipped
1781|fulton|east|frame|12|paid
1358|harbor|west|cable|85|shipped
1371|dorian|west|pump|50|shipped
2274|cobalt|south|rotor|51|paid
1904|ember|south|pump|91|paid
2267|ionic|east|valve|91|held
1984|cobalt|north|panel|47|held
1480|ionic|west|pump|10|held
2242|gale|north|panel|26|pending
2130|gale|west|rotor|33|pending
1879|juno|west|frame|17|shipped
1553|harbor|west|panel|61|paid
1672|ember|north|cable|57|shipped
1977|cobalt|west|panel|21|shipped
2069|cobalt|south|panel|46|shipped
1300|gale|south|gasket|56|pending
2167|gale|west|pump|29|pending
1834|juno|south|pump|30|held
1446|juno|north|frame|21|pending
1439|ionic|north|rotor|11|paid
1684|ionic|east|rotor|12|paid
1583|gale|east|sensor|84|paid
1634|ionic|north|valve|46|paid
1954|fulton|north|rotor|68|paid
1295|gale|south|frame|59|held
1455|juno|north|rotor|57|shipped
2181|dorian|east|panel|48|held
2088|juno|east|valve|16|held
1966|ionic|south|panel|91|shipped
1508|acme|east|gasket|98|shipped
1659|gale|east|panel|92|paid
1469|harbor|east|frame|81|pending
2192|ember|east|rotor|98|paid
1337|dorian|west|rotor|67|pending
1608|ionic|north|valve|87|held
1515|birch|south|valve|72|pending
2024|acme|east|sensor|86|pending
2036|ember|north|cable|61|pending
1488|gale|west|sensor|88|pending
1333|acme|south|gasket|16|pending
1852|acme|south|pump|46|pending
2261|birch|east|panel|32|shipped
1601|gale|south|sensor|89|held
1970|dorian|north|valve|84|shipped
1673|dorian|east|rotor|87|shipped
1726|dorian|east|cable|21|shipped
2005|gale|south|sensor|59|held
1405|fulton|south|panel|22|paid
1654|ionic|south|panel|21|paid
2226|birch|south|sensor|30|pending
2223|acme|north|pump|92|pending
2205|acme|east|panel|69|held
1307|gale|south|sensor|21|shipped
1409|gale|south|rotor|48|paid
1566|ionic|east|rotor|32|shipped
1742|gale|west|valve|24|paid
1393|acme|north|sensor|27|shipped
1313|gale|east|frame|20|pending
1987|birch|west|sensor|47|held
1955|harbor|west|sensor|43|pending
2096|cobalt|east|gasket|47|held
1594|harbor|north|gasket|24|held
2118|ember|west|cable|54|shipped
1506|acme|east|pump|44|shipped
1464|gale|north|pump|73|shipped
1280|gale|south|pump|42|paid
1411|ember|east|panel|45|pending
1649|birch|south|valve|18|shipped
1319|gale|south|gasket|86|shipped
1862|birch|south|valve|23|shipped
1949|ionic|south|gasket|44|held
2236|gale|south|rotor|30|shipped
1854|ember|west|valve|35|pending
1813|acme|east|valve|97|paid
1851|birch|south|sensor|78|pending
1473|gale|north|valve|43|held
1609|birch|south|frame|58|shipped
1925|fulton|south|sensor|81|shipped
1285|gale|south|frame|55|pending
1312|gale|south|gasket|85|pending
1756|gale|west|panel|31|paid
2186|cobalt|east|panel|88|shipped
2174|juno|west|sensor|88|shipped
2034|ionic|south|rotor|36|paid
2244|ember|north|valve|34|pending
2080|juno|north|rotor|11|pending
1669|dorian|west|cable|72|pending
1738|juno|west|valve|15|shipped
2152|gale|west|frame|16|shipped
1782|ionic|south|rotor|38|held
1627|harbor|south|cable|75|shipped
2084|ember|north|valve|54|shipped
1757|gale|west|cable|31|pending
1679|birch|south|panel|55|held
2146|acme|north|sensor|51|held
1713|acme|east|valve|31|paid
1279|gale|east|rotor|87|pending
1822|dorian|west|pump|94|pending
1592|ember|west|panel|41|pending
2232|cobalt|west|rotor|61|pending
1889|dorian|north|frame|55|held
2059|fulton|north|pump|17|shipped
1842|cobalt|east|panel|43|pending
1528|acme|north|cable|73|held
1433|cobalt|east|cable|67|pending
1346|dorian|south|panel|62|pending
1616|harbor|east|frame|84|shipped
1820|ionic|north|rotor|87|shipped
1374|juno|south|frame|61|pending
1915|ember|north|sensor|55|paid
1706|harbor|south|valve|30|held
1935|birch|north|sensor|41|held
1306|gale|north|gasket|94|pending
1437|acme|south|pump|53|paid
1572|birch|west|panel|75|paid
1598|harbor|south|cable|62|paid
2169|gale|east|sensor|70|shipped
1763|ionic|west|pump|38|paid
1899|harbor|north|pump|60|shipped
1877|ionic|east|panel|58|shipped
1702|ember|south|pump|38|shipped
1932|gale|north|rotor|78|shipped
2273|fulton|west|valve|82|pending
1882|harbor|north|cable|25|paid
1992|gale|north|cable|36|shipped
2113|ember|north|pump|85|shipped
1683|ionic|south|gasket|27|paid
1620|cobalt|east|rotor|61|held
1929|dorian|west|pump|55|shipped
1522|birch|west|valve|10|paid
1699|ionic|east|pump|56|held
1425|fulton|south|pump|10|shipped
1976|gale|east|rotor|24|shipped
1429|juno|south|rotor|67|pending
1983|gale|east|gasket|58|paid
1643|gale|west|cable|43|pending
1579|ionic|south|pump|31|pending
1353|harbor|east|sensor|80|paid
1732|cobalt|south|cable|47|shipped
1562|cobalt|west|panel|54|paid
2148|dorian|north|cable|84|shipped
1849|ionic|east|cable|93|held
1794|ember|north|cable|75|held
2137|ionic|south|rotor|20|shipped
1387|ionic|north|cable|33|shipped
1733|harbor|east|pump|50|held
1413|acme|south|rotor|57|pending
1807|harbor|south|valve|18|shipped
1892|birch|west|pump|43|paid
2140|fulton|east|sensor|40|shipped
2053|ember|north|frame|46|shipped
1922|ember|east|sensor|29|held
1662|dorian|west|pump|29|paid
2272|acme|west|cable|29|pending
1588|fulton|east|panel|20|paid
2225|acme|south|frame|90|pending
1720|juno|north|cable|91|held
1829|gale|west|panel|77|held
1382|gale|west|sensor|67|pending
2029|cobalt|south|sensor|72|shipped
2227|dorian|east|pump|11|held
2008|acme|east|rotor|23|paid
1749|fulton|south|sensor|22|pending
2060|harbor|north|panel|63|shipped
1560|fulton|south|frame|93|held
1615|fulton|east|valve|52|shipped
2212|birch|south|pump|57|pending
2048|dorian|north|sensor|99|shipped
2061|ember|east|gasket|78|pending
2168|gale|north|cable|21|pending
1764|fulton|east|rotor|21|held
2074|fulton|north|panel|51|pending
2216|gale|east|frame|82|shipped
1372|juno|east|gasket|14|paid
1323|gale|south|valve|46|pending
1787|gale|south|panel|25|pending
1363|gale|south|panel|49|paid
2000|cobalt|west|panel|52|held
1868|gale|south|cable|85|paid
1485|birch|north|frame|61|held
1418|acme|west|sensor|51|paid
1530|cobalt|north|gasket|46|held
1874|harbor|west|cable|38|paid
2089|harbor|east|panel|66|held
1548|juno|east|pump|68|paid
1324|gale|east|pump|12|pending
2157|gale|south|panel|14|paid
1836|cobalt|north|rotor|96|paid
1936|gale|south|panel|94|shipped
1570|acme|south|valve|73|paid
1646|cobalt|north|frame|31|shipped
2047|fulton|north|cable|32|pending
1428|ember|south|gasket|75|held
2161|gale|north|valve|39|paid
1500|juno|west|frame|29|shipped
1375|juno|west|frame|43|paid
2263|dorian|south|gasket|12|held
```
POLICY (apply exactly):
- Consider ONLY orders where customer = "gale" AND region = "south" AND status = "pending".
- Process those orders in ASCENDING order id.
- For each: if qty > 63, call restock(item, qty) with that order's item and qty; otherwise call cancel_order(order_id).
- Call nothing else. Every other record is irrelevant no matter how similar it looks.
Output the full ordered sequence of tool calls as a JSON array of {"tool": string, "args": object}.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
[
{"tool": "cancel_order", "args": {"order_id": 1274}},
{"tool": "cancel_order", "args": {"order_id": 1285}},
{"tool": "cancel_order", "args": {"order_id": 1300}},
{"tool": "restock", "args": {"item": "gasket", "qty": 85}},
{"tool": "cancel_order", "args": {"order_id": 1323}},
{"tool": "cancel_order", "args": {"order_id": 1787}}
]correctagentic.tools.deploy-v1conf 100% · 2.1s · $0.002 · 1184 tok
question
You are a release agent operating strictly through tool calls.
TOOL CATALOG:
- deploy(service: string) — deploys a service; FAILS if any dependency is not yet deployed
- health_check(service: string) — runs a post-deploy health probe
- rollback(service: string) — reverts a bad deploy (nothing fails in this scenario)
- scale(service: string, replicas: int) — changes replica count (out of scope)
- restart(service: string) — restarts a service (out of scope)
SERVICES AND THEIR DEPENDENCIES (a service can only be deployed after ALL its dependencies):
- reports: auth-svc
- auth-svc: (none)
- billing: reports, search
- search: auth-svc, reports
POLICY:
- Deploy in waves: in each wave, deploy every service whose dependencies are already deployed, in alphabetical order; repeat until all services are deployed.
- The service "search" is flagged unstable: call health_check on it immediately after deploying it.
- Call nothing else.
Output the full ordered sequence of tool calls needed to accomplish the goal, as a JSON array of objects {"tool": string, "args": object}. Use exactly the tool and argument names from the catalog. Do not call any tool that is not required.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
[
{"tool": "deploy", "args": {"service": "auth-svc"}},
{"tool": "deploy", "args": {"service": "reports"}},
{"tool": "deploy", "args": {"service": "search"}},
{"tool": "health_check", "args": {"service": "search"}},
{"tool": "deploy", "args": {"service": "billing"}}
]correctagentic.tools.ledger-v1conf 100% · 918ms · $0.002 · 1070 tok
question
You are a treasury agent operating strictly through tool calls.
TOOL CATALOG:
- transfer(from: string, to: string, amount: int) — moves funds between accounts; FAILS if it would overdraw the source
- top_up_from_reserve(account: string, amount: int) — adds funds to an account from the corporate reserve
- get_balance(account: string) — reads a balance (you already have all balances below — reads are unnecessary and forbidden)
- freeze_account(account: string) — compliance freeze (not part of this task)
- convert_currency(account: string, currency: string) — FX conversion (all amounts are already in USD)
OPENING BALANCES:
- echo: $805
- lima: $869
- kilo: $410
PAYMENT INSTRUCTIONS (execute in exactly this order):
1. pay $489 from "echo" to "kilo"
2. pay $492 from "echo" to "lima"
3. pay $181 from "kilo" to "echo"
4. pay $539 from "lima" to "kilo"
RULES:
- transfer() fails on overdraft. If a payment would overdraw its source account at the moment of execution, first call top_up_from_reserve() on the source with EXACTLY the shortfall (no more, no less), then execute the transfer.
- Track balances as they change: earlier payments affect later ones.
- Never call tools that are not needed.
Output the full ordered sequence of tool calls needed to accomplish the goal, as a JSON array of objects {"tool": string, "args": object}. Use exactly the tool and argument names from the catalog. Do not call any tool that is not required.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
[{"tool":"transfer","args":{"from":"echo","to":"kilo","amount":489}},{"tool":"top_up_from_reserve","args":{"account":"echo","amount":176}},{"tool":"transfer","args":{"from":"echo","to":"lima","amount":492}},{"tool":"transfer","args":{"from":"kilo","to":"echo","amount":181}},{"tool":"transfer","args":{"from":"lima","to":"kilo","amount":539}}]correctagentic.tools.context-load-v1conf 100% · 1.4s · $0.005 · 2039 tok
question
You are an order-operations agent working strictly through tool calls.
TOOL CATALOG:
- restock(item: string, qty: int) — reorders stock for a large pending order
- cancel_order(order_id: int) — cancels a small pending order
- ship_order(order_id: int) — ships a paid order (out of scope here)
- refund(order_id: int, amount: int) — refunds a customer (out of scope here)
- notify_customer(customer: string, message: string) — sends a notification (not required by this policy)
ORDER LEDGER (277 records, format: id|customer|region|item|qty|status):
```
1314|acme|west|rotor|34|pending
1909|birch|north|frame|64|held
1431|acme|east|valve|31|pending
1645|gale|west|pump|79|pending
2291|dorian|north|sensor|54|paid
1652|ember|east|frame|42|held
1319|acme|south|pump|81|pending
1762|dorian|south|cable|33|held
1927|harbor|north|panel|13|pending
2363|birch|south|cable|87|shipped
1925|ember|east|pump|19|held
1954|cobalt|east|rotor|52|held
1963|ember|north|sensor|32|pending
1311|acme|west|cable|68|shipped
1777|ember|north|cable|41|shipped
1322|acme|west|gasket|99|shipped
1392|cobalt|east|panel|36|paid
2240|ionic|east|pump|62|pending
1371|ionic|south|valve|89|paid
1459|acme|east|gasket|14|shipped
1857|fulton|west|valve|98|held
1825|cobalt|south|sensor|58|pending
1689|dorian|north|pump|24|pending
1387|juno|west|sensor|28|shipped
1427|acme|north|frame|58|shipped
1870|dorian|east|panel|73|paid
2349|juno|west|valve|24|pending
1683|fulton|north|cable|31|shipped
2368|gale|west|valve|86|shipped
2331|harbor|south|sensor|76|paid
1453|ember|north|gasket|57|paid
1526|harbor|south|gasket|29|paid
2007|juno|south|sensor|31|held
2136|dorian|east|rotor|34|paid
2033|ember|east|gasket|74|held
1407|birch|south|pump|32|held
1378|dorian|east|pump|56|held
1767|ionic|north|panel|81|held
1469|cobalt|north|pump|26|held
1478|birch|west|pump|50|pending
1539|birch|east|sensor|58|paid
1887|ionic|south|frame|55|paid
1583|birch|south|valve|72|paid
1788|gale|east|frame|41|paid
2026|juno|north|pump|90|shipped
1562|acme|east|valve|58|shipped
1568|cobalt|north|gasket|21|pending
2362|harbor|south|panel|19|shipped
1634|ember|west|gasket|21|held
1722|juno|north|sensor|38|held
1958|cobalt|south|cable|15|pending
1425|juno|east|rotor|20|shipped
1798|ember|west|sensor|30|shipped
1882|dorian|east|panel|41|pending
1720|cobalt|east|panel|70|paid
2102|juno|west|valve|44|pending
2208|fulton|west|frame|39|paid
1900|juno|south|pump|20|held
1333|acme|east|sensor|23|pending
2010|dorian|south|valve|17|pending
1587|dorian|west|valve|40|pending
2221|juno|north|pump|81|held
2189|fulton|east|gasket|80|paid
1639|fulton|west|rotor|25|paid
1729|ionic|south|rotor|38|paid
1456|fulton|west|cable|75|pending
2275|birch|north|rotor|39|held
1757|ionic|south|pump|67|paid
1592|ionic|west|frame|46|pending
1884|gale|east|frame|26|pending
1908|ionic|south|pump|74|held
2139|ember|south|sensor|21|paid
2155|harbor|south|valve|71|held
1708|harbor|east|pump|65|held
1725|acme|east|pump|93|held
2364|gale|east|cable|49|pending
1626|fulton|west|frame|74|pending
2069|juno|west|panel|81|paid
2272|dorian|south|sensor|92|pending
1955|dorian|east|panel|84|pending
2040|ember|east|sensor|64|shipped
1891|birch|south|valve|48|held
1752|cobalt|north|cable|86|pending
1794|fulton|north|panel|15|shipped
1522|dorian|north|pump|81|paid
2016|gale|east|pump|27|shipped
1841|fulton|south|valve|34|paid
2027|dorian|north|frame|95|shipped
1932|ember|north|cable|96|pending
2385|harbor|north|cable|34|shipped
1490|ember|west|panel|85|held
2366|fulton|west|frame|56|shipped
1658|juno|north|sensor|77|shipped
1570|fulton|east|panel|42|pending
2161|juno|west|valve|82|paid
1970|ionic|north|frame|68|held
1550|juno|west|valve|94|held
1680|harbor|north|rotor|47|pending
2196|ionic|east|panel|59|paid
1650|fulton|east|gasket|83|shipped
1699|ionic|north|rotor|79|held
1448|birch|north|sensor|27|pending
1947|birch|west|frame|35|paid
2104|dorian|west|cable|54|paid
2216|juno|east|pump|33|held
1474|cobalt|south|cable|36|pending
1489|fulton|north|frame|82|shipped
2066|cobalt|south|rotor|57|paid
1886|juno|east|gasket|12|pending
2073|ionic|east|rotor|84|shipped
2153|fulton|north|valve|49|paid
2379|harbor|south|rotor|23|shipped
2344|fulton|south|rotor|96|shipped
1408|ember|north|rotor|87|pending
2060|cobalt|south|panel|12|pending
1517|cobalt|north|sensor|82|paid
1669|cobalt|north|panel|96|held
1751|ember|west|panel|46|paid
2295|birch|west|sensor|30|paid
2142|birch|north|pump|10|held
1834|ember|west|gasket|75|shipped
1858|gale|east|gasket|66|shipped
1386|gale|north|frame|28|paid
2022|harbor|east|cable|78|held
1603|ionic|north|valve|43|pending
1875|acme|south|sensor|46|pending
2213|gale|west|rotor|90|shipped
1593|dorian|west|frame|23|paid
2124|ember|west|frame|36|pending
2146|gale|north|rotor|41|held
2151|harbor|east|rotor|64|held
1795|gale|east|frame|95|held
1975|acme|north|gasket|99|shipped
2253|birch|east|cable|34|pending
1424|fulton|north|rotor|52|pending
1586|cobalt|north|rotor|27|pending
2266|dorian|south|sensor|71|held
1496|gale|west|cable|55|held
1613|gale|east|rotor|37|pending
2020|ionic|south|frame|12|held
1804|gale|west|panel|57|paid
2248|ember|east|pump|43|pending
1916|cobalt|north|pump|98|paid
1414|dorian|south|pump|30|paid
2290|juno|east|frame|32|pending
2177|dorian|east|sensor|89|held
2327|birch|west|panel|18|pending
1340|acme|west|rotor|38|held
1771|harbor|north|valve|27|paid
2369|gale|north|panel|93|shipped
2302|ionic|north|sensor|23|paid
1893|acme|east|gasket|92|paid
2091|juno|east|gasket|46|pending
1676|ember|east|panel|88|paid
1354|dorian|north|panel|13|paid
2029|dorian|east|rotor|34|pending
1723|acme|south|gasket|49|shipped
1979|acme|east|panel|91|pending
2054|fulton|west|frame|17|paid
1636|cobalt|east|cable|53|held
1305|acme|north|valve|64|pending
1971|birch|north|cable|95|held
2166|juno|east|gasket|59|held
1673|birch|north|sensor|68|shipped
1344|acme|west|rotor|92|pending
1845|harbor|east|frame|45|pending
2338|acme|west|panel|90|paid
1635|harbor|north|pump|50|paid
2300|harbor|south|gasket|94|paid
1852|cobalt|south|gasket|46|pending
2014|ionic|south|rotor|35|shipped
1816|harbor|south|panel|72|held
1868|ionic|south|gasket|56|paid
2260|harbor|west|sensor|11|pending
1685|cobalt|west|frame|69|paid
1842|cobalt|east|sensor|20|shipped
1369|birch|west|valve|54|held
1511|cobalt|south|frame|68|shipped
1787|cobalt|east|rotor|41|shipped
1579|fulton|east|gasket|34|shipped
2080|gale|north|sensor|50|shipped
1582|gale|north|sensor|14|pending
1604|juno|east|valve|85|pending
2116|dorian|south|gasket|67|pending
2212|cobalt|south|gasket|51|pending
1704|fulton|north|frame|27|shipped
1830|birch|east|panel|39|pending
1501|acme|south|valve|86|held
1328|acme|west|sensor|71|pending
2025|acme|south|gasket|44|pending
2321|harbor|north|frame|29|shipped
2008|cobalt|east|frame|64|paid
2129|acme|east|pump|37|pending
1809|dorian|north|valve|47|shipped
2084|ionic|north|sensor|92|held
1750|ionic|north|frame|98|shipped
2389|cobalt|west|rotor|98|held
1671|ember|east|gasket|87|pending
1617|cobalt|east|rotor|18|paid
2004|acme|west|frame|36|shipped
1974|cobalt|east|rotor|83|paid
1864|fulton|east|rotor|32|paid
1743|acme|south|sensor|70|paid
1782|harbor|west|frame|44|paid
2125|gale|east|pump|56|pending
2376|ember|east|pump|99|held
1572|birch|west|panel|31|paid
1364|birch|north|frame|64|shipped
2304|ember|west|rotor|80|held
2094|ionic|west|frame|74|shipped
2235|juno|north|sensor|65|paid
1561|dorian|north|sensor|36|paid
2279|ionic|east|sensor|77|shipped
2355|dorian|north|cable|37|shipped
1360|acme|west|gasket|38|held
1696|gale|north|sensor|22|paid
2348|acme|south|sensor|38|pending
1443|gale|west|panel|46|held
1621|birch|east|rotor|27|shipped
1664|birch|west|rotor|10|shipped
1631|juno|east|gasket|71|paid
1396|gale|west|cable|45|shipped
2182|ionic|south|panel|38|pending
1983|dorian|west|pump|94|pending
2123|cobalt|south|sensor|70|shipped
1986|juno|north|sensor|15|held
1546|gale|north|pump|33|paid
1736|fulton|north|gasket|31|shipped
1507|juno|south|sensor|84|pending
1502|gale|west|panel|20|shipped
2228|birch|north|frame|37|held
1420|birch|west|cable|68|pending
2096|juno|east|cable|61|shipped
1896|dorian|west|gasket|69|shipped
1437|acme|south|panel|73|paid
1611|gale|west|rotor|10|shipped
2201|ionic|east|cable|67|shipped
1557|juno|east|sensor|78|held
1907|acme|east|sensor|96|paid
1933|cobalt|north|rotor|31|held
2176|fulton|west|gasket|39|pending
1807|cobalt|north|sensor|15|shipped
1528|dorian|south|frame|26|paid
2325|dorian|north|pump|57|shipped
1463|birch|north|panel|63|paid
1506|ember|south|sensor|41|shipped
1889|juno|west|rotor|60|pending
2157|birch|north|valve|84|shipped
1381|juno|south|valve|68|held
2111|juno|south|valve|39|pending
1538|fulton|east|valve|51|pending
1298|acme|west|panel|45|pending
2285|acme|north|pump|66|held
1401|birch|west|frame|92|pending
1972|fulton|north|valve|32|paid
1353|acme|west|rotor|71|shipped
2241|cobalt|east|sensor|93|shipped
1533|cobalt|north|gasket|15|paid
1997|acme|east|panel|50|held
1715|ember|south|rotor|49|paid
2308|cobalt|north|valve|52|shipped
1347|acme|north|rotor|13|pending
1484|dorian|south|panel|97|shipped
2334|juno|east|frame|99|held
1741|juno|north|cable|38|paid
1375|juno|south|valve|99|held
2047|ionic|north|frame|46|paid
1940|cobalt|north|pump|80|pending
1598|ionic|west|valve|31|shipped
2314|birch|south|gasket|29|shipped
2171|dorian|east|frame|80|paid
1902|dorian|east|panel|81|pending
1479|harbor|south|gasket|75|pending
1990|fulton|north|cable|39|paid
1920|fulton|east|rotor|34|held
1524|dorian|south|rotor|59|held
1818|cobalt|west|sensor|24|paid
```
POLICY (apply exactly):
- Consider ONLY orders where customer = "acme" AND region = "west" AND status = "pending".
- Process those orders in ASCENDING order id.
- For each: if qty > 56, call restock(item, qty) with that order's item and qty; otherwise call cancel_order(order_id).
- Call nothing else. Every other record is irrelevant no matter how similar it looks.
Output the full ordered sequence of tool calls as a JSON array of {"tool": string, "args": object}.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
[
{"tool": "cancel_order", "args": {"order_id": 1298}},
{"tool": "cancel_order", "args": {"order_id": 1314}},
{"tool": "restock", "args": {"item": "sensor", "qty": 71}},
{"tool": "restock", "args": {"item": "rotor", "qty": 92}}
]correctagentic.tools.triage-v1conf 100% · 1.5s · $0.002 · 1111 tok
question
You operate a support desk strictly through tool calls.
TOOL CATALOG:
- create_ticket(title: string, priority: int) — opens a ticket; IDs are assigned sequentially: the 1st created ticket is "TCK-1", the 2nd "TCK-2", etc.
- assign(ticket_id: string, agent: string) — assigns an open ticket to an agent
- escalate(ticket_id: string) — marks a ticket as escalated
- close_ticket(ticket_id: string, resolution: string) — closes a ticket with a resolution note
- send_email(to: string, body: string) — sends an email (not part of the triage policy)
- archive_ticket(ticket_id: string) — archives a closed ticket (nightly job does this automatically)
- set_reminder(ticket_id: string, hours: int) — sets a follow-up reminder
ROUTING POLICY (apply exactly, in this order, for each incident, processing incidents in the order listed):
1. Create a ticket for the incident (title = the incident text verbatim, priority as given).
2. If the incident is an exact duplicate of an earlier incident in this list, close its ticket immediately with resolution "duplicate of <ID of the earlier ticket>" and do nothing else for it.
3. Otherwise, if priority ≥ 9, escalate the ticket BEFORE assigning it.
4. Assign the ticket to the agent responsible for its category.
CATEGORY → AGENT:
- payments → okafor
- data → silva
- auth → tanaka
INCIDENTS:
1. "invoice total wrong" (category: payments, priority 6)
2. "dashboard shows stale numbers" (category: data, priority 4)
3. "invoice total wrong" (category: payments, priority 6)
Output the full ordered sequence of tool calls needed to accomplish the goal, as a JSON array of objects {"tool": string, "args": object}. Use exactly the tool and argument names from the catalog. Do not call any tool that is not required.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
[
{"tool": "create_ticket", "args": {"title": "invoice total wrong", "priority": 6}},
{"tool": "assign", "args": {"ticket_id": "TCK-1", "agent": "okafor"}},
{"tool": "create_ticket", "args": {"title": "dashboard shows stale numbers", "priority": 4}},
{"tool": "assign", "args": {"ticket_id": "TCK-2", "agent": "silva"}},
{"tool": "create_ticket", "args": {"title": "invoice total wrong", "priority": 6}},
{"tool": "close_ticket", "args": {"ticket_id": "TCK-3", "resolution": "duplicate of TCK-1"}}
]correctagentic.tools.deploy-v1conf 100% · 1.1s · $0.003 · 1296 tok
question
You are a release agent operating strictly through tool calls.
TOOL CATALOG:
- deploy(service: string) — deploys a service; FAILS if any dependency is not yet deployed
- health_check(service: string) — runs a post-deploy health probe
- rollback(service: string) — reverts a bad deploy (nothing fails in this scenario)
- scale(service: string, replicas: int) — changes replica count (out of scope)
- restart(service: string) — restarts a service (out of scope)
SERVICES AND THEIR DEPENDENCIES (a service can only be deployed after ALL its dependencies):
- gateway: search
- notifier: auth-svc, search
- search: (none)
- auth-svc: search
POLICY:
- Deploy in waves: in each wave, deploy every service whose dependencies are already deployed, in alphabetical order; repeat until all services are deployed.
- The service "search" is flagged unstable: call health_check on it immediately after deploying it.
- Call nothing else.
Output the full ordered sequence of tool calls needed to accomplish the goal, as a JSON array of objects {"tool": string, "args": object}. Use exactly the tool and argument names from the catalog. Do not call any tool that is not required.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
[
{"tool": "deploy", "args": {"service": "search"}},
{"tool": "health_check", "args": {"service": "search"}},
{"tool": "deploy", "args": {"service": "auth-svc"}},
{"tool": "deploy", "args": {"service": "gateway"}},
{"tool": "deploy", "args": {"service": "notifier"}}
]correctagentic.tools.ledger-v1conf 100% · 744ms · $0.003 · 1266 tok
question
You are a treasury agent operating strictly through tool calls.
TOOL CATALOG:
- transfer(from: string, to: string, amount: int) — moves funds between accounts; FAILS if it would overdraw the source
- top_up_from_reserve(account: string, amount: int) — adds funds to an account from the corporate reserve
- get_balance(account: string) — reads a balance (you already have all balances below — reads are unnecessary and forbidden)
- freeze_account(account: string) — compliance freeze (not part of this task)
- convert_currency(account: string, currency: string) — FX conversion (all amounts are already in USD)
OPENING BALANCES:
- kilo: $254
- tango: $235
- delta: $667
PAYMENT INSTRUCTIONS (execute in exactly this order):
1. pay $470 from "tango" to "delta"
2. pay $312 from "delta" to "kilo"
3. pay $335 from "tango" to "delta"
4. pay $99 from "delta" to "kilo"
RULES:
- transfer() fails on overdraft. If a payment would overdraw its source account at the moment of execution, first call top_up_from_reserve() on the source with EXACTLY the shortfall (no more, no less), then execute the transfer.
- Track balances as they change: earlier payments affect later ones.
- Never call tools that are not needed.
Output the full ordered sequence of tool calls needed to accomplish the goal, as a JSON array of objects {"tool": string, "args": object}. Use exactly the tool and argument names from the catalog. Do not call any tool that is not required.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
[
{"tool": "top_up_from_reserve", "args": {"account": "tango", "amount": 235}},
{"tool": "transfer", "args": {"from": "tango", "to": "delta", "amount": 470}},
{"tool": "transfer", "args": {"from": "delta", "to": "kilo", "amount": 312}},
{"tool": "top_up_from_reserve", "args": {"account": "tango", "amount": 335}},
{"tool": "transfer", "args": {"from": "tango", "to": "delta", "amount": 335}},
{"tool": "transfer", "args": {"from": "delta", "to": "kilo", "amount": 99}}
]correctagentic.tools.triage-v1conf 100% · 550ms · $0.003 · 1565 tok
question
You operate a support desk strictly through tool calls.
TOOL CATALOG:
- create_ticket(title: string, priority: int) — opens a ticket; IDs are assigned sequentially: the 1st created ticket is "TCK-1", the 2nd "TCK-2", etc.
- assign(ticket_id: string, agent: string) — assigns an open ticket to an agent
- escalate(ticket_id: string) — marks a ticket as escalated
- close_ticket(ticket_id: string, resolution: string) — closes a ticket with a resolution note
- send_email(to: string, body: string) — sends an email (not part of the triage policy)
- archive_ticket(ticket_id: string) — archives a closed ticket (nightly job does this automatically)
- set_reminder(ticket_id: string, hours: int) — sets a follow-up reminder
ROUTING POLICY (apply exactly, in this order, for each incident, processing incidents in the order listed):
1. Create a ticket for the incident (title = the incident text verbatim, priority as given).
2. If the incident is an exact duplicate of an earlier incident in this list, close its ticket immediately with resolution "duplicate of <ID of the earlier ticket>" and do nothing else for it.
3. Otherwise, if priority ≥ 7, escalate the ticket BEFORE assigning it.
4. Assign the ticket to the agent responsible for its category.
CATEGORY → AGENT:
- infra → okafor
- auth → dubois
- data → tanaka
INCIDENTS:
1. "uploads failing intermittently" (category: infra, priority 4)
2. "cannot reset password" (category: auth, priority 4)
3. "dashboard shows stale numbers" (category: data, priority 4)
4. "dashboard shows stale numbers" (category: data, priority 4)
Output the full ordered sequence of tool calls needed to accomplish the goal, as a JSON array of objects {"tool": string, "args": object}. Use exactly the tool and argument names from the catalog. Do not call any tool that is not required.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
[
{"tool": "create_ticket", "args": {"title": "uploads failing intermittently", "priority": 4}},
{"tool": "assign", "args": {"ticket_id": "TCK-1", "agent": "okafor"}},
{"tool": "create_ticket", "args": {"title": "cannot reset password", "priority": 4}},
{"tool": "assign", "args": {"ticket_id": "TCK-2", "agent": "dubois"}},
{"tool": "create_ticket", "args": {"title": "dashboard shows stale numbers", "priority": 4}},
{"tool": "assign", "args": {"ticket_id": "TCK-3", "agent": "tanaka"}},
{"tool": "create_ticket", "args": {"title": "dashboard shows stale numbers", "priority": 4}},
{"tool": "close_ticket", "args": {"ticket_id": "TCK-4", "resolution": "duplicate of TCK-3"}}
]correctagentic.tools.context-load-v1conf 100% · 1.3s · $0.005 · 1829 tok
question
You are an order-operations agent working strictly through tool calls.
TOOL CATALOG:
- restock(item: string, qty: int) — reorders stock for a large pending order
- cancel_order(order_id: int) — cancels a small pending order
- ship_order(order_id: int) — ships a paid order (out of scope here)
- refund(order_id: int, amount: int) — refunds a customer (out of scope here)
- notify_customer(customer: string, message: string) — sends a notification (not required by this policy)
ORDER LEDGER (220 records, format: id|customer|region|item|qty|status):
```
1745|harbor|south|cable|16|paid
1898|dorian|west|rotor|60|shipped
2158|juno|east|rotor|34|held
1566|gale|north|sensor|63|pending
2130|dorian|north|panel|98|pending
2033|juno|east|panel|39|held
1630|acme|north|panel|64|held
1483|harbor|south|cable|83|pending
2167|ionic|south|frame|40|held
1591|ember|east|valve|40|held
1823|ember|west|gasket|97|held
2200|ember|east|sensor|97|held
1942|harbor|north|gasket|31|paid
1486|dorian|west|pump|77|paid
1472|ionic|west|rotor|24|held
1600|ionic|west|pump|24|shipped
2012|harbor|east|pump|50|held
2153|fulton|south|pump|24|paid
2193|acme|west|panel|92|held
1454|cobalt|east|cable|36|pending
1695|dorian|south|panel|90|shipped
1413|ember|east|rotor|60|paid
1681|ionic|east|valve|90|shipped
1502|harbor|east|valve|98|pending
1530|dorian|east|panel|48|shipped
2184|ionic|north|gasket|98|paid
2066|acme|west|gasket|75|shipped
1584|harbor|west|panel|77|shipped
1804|harbor|east|gasket|11|held
1723|fulton|east|pump|75|pending
1659|acme|west|gasket|58|held
2205|harbor|south|pump|81|held
1563|dorian|south|rotor|41|held
1797|ember|north|cable|49|paid
1883|acme|west|pump|59|shipped
1554|cobalt|north|rotor|72|pending
1861|birch|west|frame|50|shipped
1874|harbor|south|gasket|94|pending
1864|ember|north|cable|26|held
1420|ionic|north|cable|94|held
1619|harbor|south|rotor|61|pending
1721|ionic|south|gasket|50|shipped
1633|fulton|east|sensor|33|pending
1752|cobalt|south|gasket|96|shipped
1989|dorian|west|panel|72|pending
1635|ember|west|valve|91|paid
1575|ionic|east|cable|48|shipped
1792|ember|south|pump|59|pending
2109|dorian|north|panel|63|pending
1988|cobalt|north|valve|43|pending
1971|cobalt|north|cable|38|paid
2090|dorian|north|gasket|16|paid
1627|acme|south|sensor|48|paid
2183|fulton|south|cable|54|paid
1779|harbor|west|sensor|67|paid
2216|dorian|west|frame|45|shipped
1648|acme|north|sensor|41|pending
1515|ionic|east|valve|21|held
1808|juno|north|pump|51|shipped
1747|acme|east|rotor|97|shipped
2147|fulton|east|cable|92|pending
1446|cobalt|east|rotor|48|held
1663|fulton|north|cable|86|pending
1836|juno|south|cable|51|held
2135|ember|north|valve|72|held
2052|harbor|south|sensor|62|paid
2010|gale|north|frame|57|shipped
2131|cobalt|east|valve|91|shipped
1742|ember|west|sensor|89|held
2045|birch|north|frame|70|held
1949|juno|south|valve|71|pending
1905|cobalt|north|gasket|73|paid
1766|gale|east|frame|41|paid
1581|harbor|east|frame|99|pending
1614|ember|south|panel|16|held
1916|ionic|west|sensor|73|pending
1679|ember|north|gasket|16|paid
1814|juno|east|pump|69|pending
1419|ember|north|rotor|43|held
1522|cobalt|east|sensor|99|paid
1850|birch|west|cable|23|shipped
1465|juno|east|gasket|23|pending
1739|cobalt|south|panel|36|held
1719|birch|east|frame|15|held
1570|fulton|east|valve|21|pending
2020|ember|east|cable|34|held
1844|ionic|west|frame|47|held
1687|juno|west|frame|57|pending
1699|acme|west|panel|38|shipped
2220|ember|north|cable|50|paid
1704|juno|north|pump|31|shipped
2005|cobalt|west|rotor|26|pending
1436|dorian|east|gasket|41|held
1993|dorian|east|cable|77|held
1998|harbor|north|gasket|36|paid
1450|ember|south|frame|90|shipped
1552|fulton|west|valve|69|pending
1690|cobalt|south|cable|74|held
1380|ember|west|sensor|52|pending
1411|ember|north|gasket|54|shipped
1786|ember|north|panel|54|held
1730|birch|south|cable|54|pending
1595|fulton|east|valve|43|shipped
2021|juno|west|rotor|79|pending
1975|dorian|south|cable|14|paid
2162|juno|south|valve|30|shipped
1885|juno|east|gasket|41|held
1545|juno|north|gasket|23|pending
1445|harbor|south|cable|97|paid
1442|fulton|south|sensor|54|held
1604|dorian|north|panel|42|shipped
1350|ember|west|valve|30|pending
1911|cobalt|east|rotor|62|held
1358|ember|north|rotor|24|pending
1622|juno|north|frame|76|paid
1463|ionic|south|cable|44|paid
1887|acme|south|valve|80|shipped
1499|dorian|east|cable|37|held
1928|cobalt|north|rotor|19|pending
1541|ionic|north|pump|80|shipped
2116|juno|east|gasket|98|paid
1376|ember|north|sensor|87|pending
1718|acme|east|pump|57|held
1909|ionic|south|panel|12|held
2182|harbor|west|rotor|38|paid
1348|ember|north|frame|92|pending
1964|fulton|north|rotor|98|held
1427|ember|north|pump|95|shipped
2190|acme|east|rotor|31|paid
1439|dorian|north|sensor|67|held
1934|cobalt|west|frame|14|shipped
2084|acme|north|rotor|34|shipped
1925|ember|west|valve|56|held
1955|ionic|east|pump|33|held
2031|acme|north|gasket|31|held
1838|harbor|north|pump|80|pending
1938|gale|south|panel|38|paid
1647|ionic|south|valve|49|paid
2203|juno|east|sensor|31|held
2072|juno|south|cable|27|paid
1408|gale|west|sensor|70|paid
1822|harbor|east|sensor|64|pending
1433|birch|south|gasket|75|shipped
1520|fulton|east|rotor|54|paid
1746|harbor|south|gasket|29|pending
2070|juno|north|pump|14|shipped
1461|fulton|south|sensor|84|paid
2023|harbor|east|valve|95|paid
1521|fulton|west|frame|99|paid
1412|birch|north|rotor|82|paid
1641|cobalt|east|gasket|95|held
1410|birch|east|pump|61|shipped
1918|ember|south|panel|52|held
1820|cobalt|north|pump|59|pending
1958|ionic|east|rotor|58|paid
2097|ionic|south|rotor|46|pending
2178|juno|north|valve|27|pending
1534|juno|east|sensor|80|shipped
1482|fulton|south|pump|30|shipped
1509|gale|east|cable|85|held
2104|birch|south|rotor|44|pending
2211|harbor|east|valve|22|shipped
1892|dorian|north|frame|22|shipped
1529|gale|east|panel|49|pending
1921|cobalt|east|sensor|72|shipped
1498|ionic|south|panel|72|held
1369|ember|north|valve|70|shipped
2040|cobalt|north|gasket|88|pending
1455|cobalt|south|pump|49|held
1673|acme|north|panel|11|paid
1409|gale|north|rotor|97|held
2120|birch|north|gasket|17|paid
1831|acme|west|frame|88|paid
1881|fulton|east|frame|26|paid
1712|cobalt|north|frame|78|held
1667|ember|north|frame|40|pending
1613|harbor|east|frame|58|shipped
2059|harbor|south|frame|44|shipped
1579|dorian|west|rotor|90|held
1559|ionic|north|gasket|70|shipped
1982|birch|north|sensor|20|held
1353|ember|north|panel|67|shipped
1362|ember|south|panel|85|pending
2171|juno|north|gasket|95|shipped
1652|cobalt|east|sensor|68|shipped
1426|acme|south|panel|76|shipped
1653|dorian|south|rotor|26|paid
2223|dorian|east|rotor|62|paid
1846|juno|south|panel|27|pending
1869|ionic|west|frame|42|pending
2035|ember|west|frame|33|held
1395|juno|west|rotor|73|pending
1609|juno|east|pump|33|held
1827|ember|east|cable|73|shipped
1598|cobalt|west|valve|76|paid
2124|ember|north|frame|88|held
2081|gale|east|cable|58|paid
1735|cobalt|south|pump|62|shipped
1783|harbor|north|valve|90|pending
1479|cobalt|east|rotor|63|pending
1611|cobalt|west|frame|59|held
1688|cobalt|south|frame|57|paid
1553|ember|south|panel|93|paid
1632|acme|west|cable|78|shipped
2026|ionic|south|sensor|11|shipped
1492|harbor|west|gasket|52|paid
2030|acme|east|pump|47|held
2142|dorian|east|rotor|98|paid
1402|cobalt|east|gasket|46|held
1857|juno|west|rotor|33|paid
1748|harbor|west|sensor|85|paid
2056|acme|south|sensor|23|shipped
1772|birch|east|pump|94|held
1387|ember|north|pump|96|shipped
2074|juno|north|sensor|72|shipped
1706|ionic|west|rotor|81|held
1392|dorian|north|pump|86|pending
2100|dorian|north|rotor|29|pending
2014|cobalt|north|frame|21|held
1759|harbor|east|cable|46|held
```
POLICY (apply exactly):
- Consider ONLY orders where customer = "ember" AND region = "north" AND status = "pending".
- Process those orders in ASCENDING order id.
- For each: if qty > 57, call restock(item, qty) with that order's item and qty; otherwise call cancel_order(order_id).
- Call nothing else. Every other record is irrelevant no matter how similar it looks.
Output the full ordered sequence of tool calls as a JSON array of {"tool": string, "args": object}.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
[{"tool":"restock","args":{"item":"frame","qty":92}},{"tool":"cancel_order","args":{"order_id":1358}},{"tool":"restock","args":{"item":"sensor","qty":87}},{"tool":"cancel_order","args":{"order_id":1667}}]correctagentic.tools.deploy-v1conf 100% · 1.1s · $0.002 · 1051 tok
question
You are a release agent operating strictly through tool calls.
TOOL CATALOG:
- deploy(service: string) — deploys a service; FAILS if any dependency is not yet deployed
- health_check(service: string) — runs a post-deploy health probe
- rollback(service: string) — reverts a bad deploy (nothing fails in this scenario)
- scale(service: string, replicas: int) — changes replica count (out of scope)
- restart(service: string) — restarts a service (out of scope)
SERVICES AND THEIR DEPENDENCIES (a service can only be deployed after ALL its dependencies):
- auth-svc: notifier
- notifier: (none)
- billing: auth-svc
- search: auth-svc
POLICY:
- Deploy in waves: in each wave, deploy every service whose dependencies are already deployed, in alphabetical order; repeat until all services are deployed.
- The service "notifier" is flagged unstable: call health_check on it immediately after deploying it.
- Call nothing else.
Output the full ordered sequence of tool calls needed to accomplish the goal, as a JSON array of objects {"tool": string, "args": object}. Use exactly the tool and argument names from the catalog. Do not call any tool that is not required.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
[{"tool":"deploy","args":{"service":"notifier"}},{"tool":"health_check","args":{"service":"notifier"}},{"tool":"deploy","args":{"service":"auth-svc"}},{"tool":"deploy","args":{"service":"billing"}},{"tool":"deploy","args":{"service":"search"}}]correctagentic.tools.ledger-v1conf 100% · 1.2s · $0.003 · 1600 tok
question
You are a treasury agent operating strictly through tool calls.
TOOL CATALOG:
- transfer(from: string, to: string, amount: int) — moves funds between accounts; FAILS if it would overdraw the source
- top_up_from_reserve(account: string, amount: int) — adds funds to an account from the corporate reserve
- get_balance(account: string) — reads a balance (you already have all balances below — reads are unnecessary and forbidden)
- freeze_account(account: string) — compliance freeze (not part of this task)
- convert_currency(account: string, currency: string) — FX conversion (all amounts are already in USD)
OPENING BALANCES:
- bravo: $323
- kilo: $171
- lima: $505
PAYMENT INSTRUCTIONS (execute in exactly this order):
1. pay $495 from "bravo" to "kilo"
2. pay $391 from "bravo" to "kilo"
3. pay $432 from "bravo" to "lima"
4. pay $385 from "kilo" to "lima"
5. pay $328 from "lima" to "bravo"
RULES:
- transfer() fails on overdraft. If a payment would overdraw its source account at the moment of execution, first call top_up_from_reserve() on the source with EXACTLY the shortfall (no more, no less), then execute the transfer.
- Track balances as they change: earlier payments affect later ones.
- Never call tools that are not needed.
Output the full ordered sequence of tool calls needed to accomplish the goal, as a JSON array of objects {"tool": string, "args": object}. Use exactly the tool and argument names from the catalog. Do not call any tool that is not required.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
[
{"tool": "top_up_from_reserve", "args": {"account": "bravo", "amount": 172}},
{"tool": "transfer", "args": {"from": "bravo", "to": "kilo", "amount": 495}},
{"tool": "top_up_from_reserve", "args": {"account": "bravo", "amount": 391}},
{"tool": "transfer", "args": {"from": "bravo", "to": "kilo", "amount": 391}},
{"tool": "top_up_from_reserve", "args": {"account": "bravo", "amount": 432}},
{"tool": "transfer", "args": {"from": "bravo", "to": "lima", "amount": 432}},
{"tool": "transfer", "args": {"from": "kilo", "to": "lima", "amount": 385}},
{"tool": "transfer", "args": {"from": "lima", "to": "bravo", "amount": 328}}
]correctagentic.tools.triage-v1conf 100% · 1.1s · $0.003 · 1359 tok
question
You operate a support desk strictly through tool calls.
TOOL CATALOG:
- create_ticket(title: string, priority: int) — opens a ticket; IDs are assigned sequentially: the 1st created ticket is "TCK-1", the 2nd "TCK-2", etc.
- assign(ticket_id: string, agent: string) — assigns an open ticket to an agent
- escalate(ticket_id: string) — marks a ticket as escalated
- close_ticket(ticket_id: string, resolution: string) — closes a ticket with a resolution note
- send_email(to: string, body: string) — sends an email (not part of the triage policy)
- archive_ticket(ticket_id: string) — archives a closed ticket (nightly job does this automatically)
- set_reminder(ticket_id: string, hours: int) — sets a follow-up reminder
ROUTING POLICY (apply exactly, in this order, for each incident, processing incidents in the order listed):
1. Create a ticket for the incident (title = the incident text verbatim, priority as given).
2. If the incident is an exact duplicate of an earlier incident in this list, close its ticket immediately with resolution "duplicate of <ID of the earlier ticket>" and do nothing else for it.
3. Otherwise, if priority ≥ 9, escalate the ticket BEFORE assigning it.
4. Assign the ticket to the agent responsible for its category.
CATEGORY → AGENT:
- infra → okafor
- payments → rivera
- auth → novak
INCIDENTS:
1. "uploads failing intermittently" (category: infra, priority 3)
2. "card declined at checkout" (category: payments, priority 3)
3. "SSO loop on login" (category: auth, priority 5)
4. "card declined at checkout" (category: payments, priority 3)
Output the full ordered sequence of tool calls needed to accomplish the goal, as a JSON array of objects {"tool": string, "args": object}. Use exactly the tool and argument names from the catalog. Do not call any tool that is not required.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
[
{"tool": "create_ticket", "args": {"title": "uploads failing intermittently", "priority": 3}},
{"tool": "assign", "args": {"ticket_id": "TCK-1", "agent": "okafor"}},
{"tool": "create_ticket", "args": {"title": "card declined at checkout", "priority": 3}},
{"tool": "assign", "args": {"ticket_id": "TCK-2", "agent": "rivera"}},
{"tool": "create_ticket", "args": {"title": "SSO loop on login", "priority": 5}},
{"tool": "assign", "args": {"ticket_id": "TCK-3", "agent": "novak"}},
{"tool": "create_ticket", "args": {"title": "card declined at checkout", "priority": 3}},
{"tool": "close_ticket", "args": {"ticket_id": "TCK-4", "resolution": "duplicate of TCK-2"}}
]correctagentic.tools.context-load-v1anchorconf 100% · 2.5s · $0.004 · 1683 tok
model answer:
[
{"tool": "restock", "args": {"item": "gasket", "qty": 60}},
{"tool": "cancel_order", "args": {"order_id": 1368}},
{"tool": "cancel_order", "args": {"order_id": 1384}},
{"tool": "restock", "args": {"item": "frame", "qty": 58}},
{"tool": "restock", "args": {"item": "rotor", "qty": 75}},
{"tool": "cancel_order", "args": {"order_id": 1682}}
]correctagentic.tools.deploy-v1anchorconf 100% · 577ms · $0.002 · 820 tok
model answer:
[{"tool":"deploy","args":{"service":"notifier"}},{"tool":"deploy","args":{"service":"auth-svc"}},{"tool":"deploy","args":{"service":"gateway"}},{"tool":"health_check","args":{"service":"gateway"}},{"tool":"deploy","args":{"service":"reports"}}]correctagentic.tools.triage-v1anchorconf 100% · 2.9s · $0.003 · 1568 tok
model answer:
[
{"tool":"create_ticket","args":{"title":"API latency spikes","priority":9}},
{"tool":"escalate","args":{"ticket_id":"TCK-1"}},
{"tool":"assign","args":{"ticket_id":"TCK-1","agent":"tanaka"}},
{"tool":"create_ticket","args":{"title":"records missing after import","priority":2}},
{"tool":"assign","args":{"ticket_id":"TCK-2","agent":"novak"}},
{"tool":"create_ticket","args":{"title":"refund double-charged","priority":2}},
{"tool":"assign","args":{"ticket_id":"TCK-3","agent":"okafor"}},
{"tool":"create_ticket","args":{"title":"API latency spikes","priority":9}},
{"tool":"close_ticket","args":{"ticket_id":"TCK-4","resolution":"duplicate of TCK-1"}}
]correctagentic.tools.ledger-v1anchorconf 100% · 2.9s · $0.003 · 1253 tok
model answer:
[{"tool":"top_up_from_reserve","args":{"account":"delta","amount":134}},{"tool":"transfer","args":{"from":"delta","to":"bravo","amount":451}},{"tool":"transfer","args":{"from":"bravo","to":"delta","amount":292}},{"tool":"transfer","args":{"from":"delta","to":"tango","amount":97}},{"tool":"top_up_from_reserve","args":{"account":"delta","amount":345}},{"tool":"transfer","args":{"from":"delta","to":"bravo","amount":540}},{"tool":"transfer","args":{"from":"tango","to":"bravo","amount":125}}]code 30/30 correct
correctcode.trace.nested-v1conf 100% · 1.2s · $0.006 · 2890 tok
question
Trace this Python program exactly. What does it print?
```python
total = 0
for i in range(1, 8):
for j in range(1, 6):
if j == 3 and i % 2 == 0:
break
if (i + j) % 4 == 0:
continue
total += i * 5 + j
print(total)
```
End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
397correctcode.trace.js-v1conf 100% · 627ms · $0.002 · 891 tok
question
What does this JavaScript program log? ```js const arr = [10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21]; const out = arr .map(n => n * 4) .filter(n => n % 5 === 0) .reduce((a, b) => a + b, 0); console.log(out); ``` End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
180correctcode.trace.nested-v1conf 100% · 688ms · $0.006 · 2874 tok
question
Trace this Python program exactly. What does it print?
```python
total = 0
for i in range(1, 6):
for j in range(1, 5):
if j == 3 and i % 2 == 0:
break
if (i + j) % 4 == 0:
continue
total += i * 2 + j
print(total)
```
End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
101correctcode.trace.python-v1conf 100% · 1.0s · $0.002 · 1166 tok
question
Execute this Python snippet mentally. What is printed?
```python
total = 0
v = 9
while total + v <= 84:
if v % 5 != 0:
total += v
v += 3
print(total)
```
End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
84correctcode.trace.js-v1conf 100% · 1.0s · $0.001 · 355 tok
question
Evaluate the following JavaScript. What number is logged to the console? ```js const arr = [5, 6, 7, 8, 9, 10, 11, 12]; const out = arr .map(n => n * 4) .filter(n => n % 5 === 0) .reduce((a, b) => a + b, 0); console.log(out); ``` End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
60correctcode.trace.nested-v1conf 100% · 1.0s · $0.006 · 3185 tok
question
Trace this Python program exactly. What does it print?
```python
total = 0
for i in range(1, 7):
for j in range(1, 5):
if j == 3 and i % 2 == 0:
break
if (i + j) % 3 == 0:
continue
total += i * 4 + j
print(total)
```
End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
186correctcode.trace.python-v1conf 100% · 614ms · $0.002 · 1148 tok
question
What does this Python program print?
```python
total = 0
v = 15
while total + v <= 94:
if v % 3 != 0:
total += v
v += 8
print(total)
```
End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
54correctcode.trace.js-v1conf 100% · 882ms · $0.001 · 376 tok
question
Evaluate the following JavaScript. What number is logged to the console? ```js const arr = [5, 6, 7, 8, 9, 10, 11, 12]; const out = arr .map(n => n * 7) .filter(n => n % 3 === 0) .reduce((a, b) => a + b, 0); console.log(out); ``` End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
189correctcode.trace.python-v1conf 100% · 819ms · $0.004 · 1978 tok
question
Trace the following Python code and give its exact output.
```python
total = 0
v = 7
while total + v <= 112:
if v % 6 != 0:
total += v
v += 6
print(total)
```
End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
95correctcode.trace.nested-v1conf 100% · 845ms · $0.007 · 3442 tok
question
Trace this Python program exactly. What does it print?
```python
total = 0
for i in range(1, 7):
for j in range(1, 7):
if j == 5 and i % 2 == 0:
break
if (i + j) % 4 == 0:
continue
total += i * 5 + j
print(total)
```
End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
463correctcode.trace.js-v1conf 100% · 1.2s · $0.002 · 727 tok
question
Evaluate the following JavaScript. What number is logged to the console? ```js const arr = [2, 3, 4, 5, 6, 7, 8]; const out = arr .map(n => n * 6) .filter(n => n % 4 === 0) .reduce((a, b) => a + b, 0); console.log(out); ``` End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
120correctcode.trace.python-v1conf 100% · 752ms · $0.005 · 2273 tok
question
Trace the following Python code and give its exact output.
```python
total = 0
v = 7
while total + v <= 88:
if v % 3 != 0:
total += v
v += 8
print(total)
```
End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
61correctcode.trace.nested-v1conf 99% · 2.2s · $0.005 · 2645 tok
question
Trace this Python program exactly. What does it print?
```python
total = 0
for i in range(1, 5):
for j in range(1, 7):
if j == 4 and i % 2 == 0:
break
if (i + j) % 3 == 0:
continue
total += i * 4 + j
print(total)
```
End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
147correctcode.trace.js-v1conf 100% · 1.3s · $0.001 · 390 tok
question
Evaluate the following JavaScript. What number is logged to the console? ```js const arr = [4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]; const out = arr .map(n => n * 5) .filter(n => n % 2 === 0) .reduce((a, b) => a + b, 0); console.log(out); ``` End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
270correctcode.trace.python-v1conf 100% · 2.5s · $0.003 · 1511 tok
question
Execute this Python snippet mentally. What is printed?
```python
total = 0
v = 13
while total + v <= 112:
if v % 5 != 0:
total += v
v += 4
print(total)
```
End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
80correctcode.trace.nested-v1conf 100% · 574ms · $0.004 · 2045 tok
question
Trace this Python program exactly. What does it print?
```python
total = 0
for i in range(1, 6):
for j in range(1, 5):
if j == 3 and i % 2 == 0:
break
if (i + j) % 3 == 0:
continue
total += i * 2 + j
print(total)
```
End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
79correctcode.trace.js-v1conf 100% · 1.1s · $0.001 · 539 tok
question
Evaluate the following JavaScript. What number is logged to the console? ```js const arr = [6, 7, 8, 9, 10, 11, 12, 13, 14, 15]; const out = arr .map(n => n * 5) .filter(n => n % 3 === 0) .reduce((a, b) => a + b, 0); console.log(out); ``` End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
210correctcode.trace.nested-v1conf 100% · 554ms · $0.007 · 3594 tok
question
Trace this Python program exactly. What does it print?
```python
total = 0
for i in range(1, 8):
for j in range(1, 8):
if j == 6 and i % 2 == 0:
break
if (i + j) % 2 == 0:
continue
total += i * 2 + j
print(total)
```
End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
243correctcode.trace.python-v1conf 100% · 1.1s · $0.002 · 1046 tok
question
Trace the following Python code and give its exact output.
```python
total = 0
v = 13
while total + v <= 84:
if v % 7 != 0:
total += v
v += 8
print(total)
```
End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
79correctcode.trace.js-v1conf 100% · 1.1s · $0.001 · 366 tok
question
Evaluate the following JavaScript. What number is logged to the console? ```js const arr = [8, 9, 10, 11, 12, 13, 14, 15, 16, 17]; const out = arr .map(n => n * 2) .filter(n => n % 4 === 0) .reduce((a, b) => a + b, 0); console.log(out); ``` End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
120correctcode.trace.nested-v1conf 100% · 1.6s · $0.006 · 2787 tok
question
Trace this Python program exactly. What does it print?
```python
total = 0
for i in range(1, 5):
for j in range(1, 8):
if j == 5 and i % 2 == 0:
break
if (i + j) % 4 == 0:
continue
total += i * 3 + j
print(total)
```
End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
168correctcode.trace.python-v1conf 100% · 1.0s · $0.001 · 522 tok
question
Execute this Python snippet mentally. What is printed?
```python
total = 0
v = 15
while total + v <= 107:
if v % 3 != 0:
total += v
v += 7
print(total)
```
End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
94correctcode.trace.js-v1conf 100% · 529ms · $0.001 · 615 tok
question
What does this JavaScript program log? ```js const arr = [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]; const out = arr .map(n => n * 6) .filter(n => n % 4 === 0) .reduce((a, b) => a + b, 0); console.log(out); ``` End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
252correctcode.trace.python-v1conf 100% · 1.4s · $0.002 · 1167 tok
question
Execute this Python snippet mentally. What is printed?
```python
total = 0
v = 12
while total + v <= 45:
if v % 4 != 0:
total += v
v += 3
print(total)
```
End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
33correctcode.trace.nested-v1conf 99% · 913ms · $0.007 · 3676 tok
question
Trace this Python program exactly. What does it print?
```python
total = 0
for i in range(1, 8):
for j in range(1, 8):
if j == 3 and i % 2 == 0:
break
if (i + j) % 3 == 0:
continue
total += i * 5 + j
print(total)
```
End your reply with exactly two plain-text lines (no markdown, no extra text after them):
ANSWER: <your final answer only>
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
548correctcode.trace.js-v1conf 100% · 2.2s · $0.001 · 386 tok
question
What does this JavaScript program log? ```js const arr = [3, 4, 5, 6, 7, 8, 9, 10]; const out = arr .map(n => n * 5) .filter(n => n % 2 === 0) .reduce((a, b) => a + b, 0); console.log(out); ``` End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
140correctcode.trace.python-v1anchorconf 100% · 2.2s · $0.003 · 1570 tok
model answer:
0correctcode.trace.nested-v1anchorconf 100% · 1.6s · $0.004 · 2206 tok
model answer:
283correctcode.trace.js-v1anchorconf 100% · 571ms · $0.001 · 450 tok
model answer:
63correctcode.trace.python-v1anchorconf 100% · 2.1s · $0.002 · 1139 tok
model answer:
40instruction following 29/30 correct
correctif.constraints.stack-v1conf 100% · 816ms · $0.003 · 1289 tok
question
Write in English about winter mornings, following ALL of these rules simultaneously: 1. Exactly 15 words. 2. The first word must be "flint" and the last word must be "orbit". 3. Use the word "basalt" exactly 3 times (in addition to rules 2 if they differ). 4. The letter "q" must not appear anywhere. 5. Everything entirely in lowercase. Give the text itself as your answer. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
flint dusts basalt frosted basalt cold basalt as pale sun begins its soft regular orbitcorrectif.format.acronym-v1conf 100% · 1.8s · $0.001 · 408 tok
question
Take the second letter of each of these words, in order: flint, echo, basalt, lumen. Concatenate them in uppercase into a single string with no separators. Output that string as your answer. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
LCAUcorrectif.constraints.stack-v1conf 100% · 1.2s · $0.002 · 910 tok
question
Write in English about a long journey, following ALL of these rules simultaneously: 1. Exactly 20 words. 2. The first word must be "ember" and the last word must be "delta". 3. Use the word "prism" exactly 2 times (in addition to rules 2 if they differ). 4. The letter "q" must not appear anywhere. 5. Everything entirely in lowercase. Give the text itself as your answer. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
ember guided our small cart past one glowing prism then a second sharp prism across sunbaked plains toward lush deltacorrectif.format.repeat-v1conf 100% · 1.1s · $0.001 · 472 tok
question
Write the word "falcon" in uppercase form, repeated exactly 8 times, joined by the character "_" with no spaces. Output that string as your answer. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
FALCON_FALCON_FALCON_FALCON_FALCON_FALCON_FALCON_FALCONcorrectif.constraints.stack-v1conf 100% · 958ms · $0.003 · 1440 tok
question
Write in English about an old machine, following ALL of these rules simultaneously: 1. Exactly 19 words. 2. The first word must be "lumen" and the last word must be "nova". 3. Use the word "cedar" exactly 2 times (in addition to rules 2 if they differ). 4. The letter "q" must not appear anywhere. 5. Everything entirely in lowercase. Give the text itself as your answer. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
lumen flickered over smooth cedar dented cedar workbench as the vintage projector breathed out one final bright cosmic novacorrectif.format.acronym-v1conf 100% · 1.2s · $0.001 · 330 tok
question
Take the third letter of each of these words, in order: orbit, nova, basalt, echo. Concatenate them in uppercase into a single string with no separators. Output that string as your answer. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
BVSHcorrectif.format.repeat-v1conf 100% · 787ms · $0.000 · 232 tok
question
Write the word "ember" in capitalized form, repeated exactly 5 times, joined by the character "-" with no spaces. Output that string as your answer. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
EMBER-EMBER-EMBER-EMBER-EMBERcorrectif.constraints.stack-v1conf 100% · 1.1s · $0.002 · 1166 tok
question
Write in English about an old machine, following ALL of these rules simultaneously: 1. Exactly 15 words. 2. The first word must be "lumen" and the last word must be "ember". 3. Use the word "drift" exactly 3 times (in addition to rules 2 if they differ). 4. The letter "q" must not appear anywhere. 5. Everything entirely in lowercase. Give the text itself as your answer. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
lumen motes drift drift drift slowly across cool dark rooms toward a single smoldering embercorrectif.format.acronym-v1conf 100% · 1.2s · $0.001 · 475 tok
question
Take the third letter of each of these words, in order: prism, falcon, orbit, drift, comet, flint. Concatenate them in uppercase into a single string with no separators. Output that string as your answer. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
ILBIMIcorrectif.format.repeat-v1conf 100% · 1.1s · $0.000 · 171 tok
question
Write the word "orbit" in lowercase form, repeated exactly 4 times, joined by the character "-" with no spaces. Output that string as your answer. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
orbit-orbit-orbit-orbitcorrectif.constraints.stack-v1conf 100% · 1.6s · $0.002 · 770 tok
question
Write in English about winter mornings, following ALL of these rules simultaneously: 1. Exactly 15 words. 2. The first word must be "delta" and the last word must be "ember". 3. Use the word "comet" exactly 3 times (in addition to rules 2 if they differ). 4. The letter "q" must not appear anywhere. 5. Everything entirely in lowercase. Give the text itself as your answer. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
delta soft comet breathes bright comet floats slow sweet comet curls close to warm embercorrectif.format.acronym-v1conf 100% · 1.8s · $0.000 · 162 tok
question
Take the first letter of each of these words, in order: drift, tundra, basalt, ember. Concatenate them in uppercase into a single string with no separators. Output that string as your answer. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
DTBEcorrectif.format.repeat-v1conf 100% · 1.0s · $0.000 · 205 tok
question
Write the word "zephyr" in uppercase form, repeated exactly 4 times, joined by the character "/" with no spaces. Output that string as your answer. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
ZEPHYR/ZEPHYR/ZEPHYR/ZEPHYRcorrectif.format.acronym-v1conf 100% · 750ms · $0.001 · 481 tok
question
Take the second letter of each of these words, in order: zephyr, flint, quartz, echo, nova, lumen. Concatenate them in uppercase into a single string with no separators. Output that string as your answer. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
ELUCOUcorrectif.constraints.stack-v1conf 100% · 1.2s · $0.002 · 926 tok
question
Write in English about a city at night, following ALL of these rules simultaneously: 1. Exactly 23 words. 2. The first word must be "flint" and the last word must be "basalt". 3. Use the word "nova" exactly 3 times (in addition to rules 2 if they differ). 4. The letter "q" must not appear anywhere. 5. Everything entirely in lowercase. Give the text itself as your answer. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
flint curbs catch the flare of nova nova nova neon signs that paint bright hues across weathered solid ancient heavy black natural basaltcorrectif.format.repeat-v1conf 100% · 544ms · $0.001 · 321 tok
question
Write the word "nova" in uppercase form, repeated exactly 9 times, joined by the character "/" with no spaces. Output that string as your answer. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
NOVA/NOVA/NOVA/NOVA/NOVA/NOVA/NOVA/NOVA/NOVAcorrectif.constraints.stack-v1conf 100% · 663ms · $0.003 · 1332 tok
question
Write in English about a city at night, following ALL of these rules simultaneously: 1. Exactly 23 words. 2. The first word must be "delta" and the last word must be "echo". 3. Use the word "falcon" exactly 2 times (in addition to rules 2 if they differ). 4. The letter "q" must not appear anywhere. 5. Everything entirely in lowercase. Give the text itself as your answer. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
delta warm urban night sprawls out while one falcon gives chase to another falcon above cafes that hum with low soft living echowrongif.format.acronym-v1conf 100% · 2.8s · $0.001 · 297 tok
question
Take the second letter of each of these words, in order: delta, tundra, lumen, prism, comet. Concatenate them in uppercase into a single string with no separators. Output that string as your answer. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
EUROcorrectif.format.repeat-v1conf 100% · 855ms · $0.000 · 181 tok
question
Write the word "echo" in lowercase form, repeated exactly 5 times, joined by the character "_" with no spaces. Output that string as your answer. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
echo_echo_echo_echo_echocorrectif.format.acronym-v1conf 100% · 1.2s · $0.001 · 356 tok
question
Take the first letter of each of these words, in order: drift, prism, delta, basalt, zephyr, quartz. Concatenate them in uppercase into a single string with no separators. Output that string as your answer. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
DPDBZQcorrectif.constraints.stack-v1conf 100% · 641ms · $0.002 · 966 tok
question
Write in English about the sea, following ALL of these rules simultaneously: 1. Exactly 19 words. 2. The first word must be "drift" and the last word must be "nova". 3. Use the word "quartz" exactly 2 times (in addition to rules 2 if they differ). 4. The letter "j" must not appear anywhere. 5. Everything entirely in lowercase. Give the text itself as your answer. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
drift over weathered quartz rocks and polished quartz sands beneath skies that unfurl to greet the burning stellar novacorrectif.format.repeat-v1conf 100% · 1.3s · $0.001 · 319 tok
question
Write the word "echo" in capitalized form, repeated exactly 6 times, joined by the character "-" with no spaces. Output that string as your answer. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
ECHO-ECHO-ECHO-ECHO-ECHO-ECHOcorrectif.format.acronym-v1conf 100% · 881ms · $0.001 · 664 tok
question
Take the third letter of each of these words, in order: flint, echo, comet, tundra, nova. Concatenate them in uppercase into a single string with no separators. Output that string as your answer. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
IHMNVcorrectif.constraints.stack-v1conf 100% · 1.2s · $0.002 · 1058 tok
question
Write in English about a long journey, following ALL of these rules simultaneously: 1. Exactly 15 words. 2. The first word must be "orbit" and the last word must be "ember". 3. Use the word "echo" exactly 2 times (in addition to rules 2 if they differ). 4. The letter "q" must not appear anywhere. 5. Everything entirely in lowercase. Give the text itself as your answer. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
orbit our old probes echo soft tones that echo past the dying suns last embercorrectif.format.repeat-v1conf 100% · 1.1s · $0.001 · 387 tok
question
Write the word "lumen" in lowercase form, repeated exactly 8 times, joined by the character "-" with no spaces. Output that string as your answer. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
lumen-lumen-lumen-lumen-lumen-lumen-lumen-lumencorrectif.constraints.stack-v1anchorconf 100% · 926ms · $0.003 · 1319 tok
model answer:
drift over rain-slicked asphalt under stringed lumen lights that pour soft lumen threads around small bright lumen sparks that continually loop and enter orbitcorrectif.format.acronym-v1conf 100% · 542ms · $0.001 · 577 tok
question
Take the second letter of each of these words, in order: prism, ember, nova, cedar, quartz, comet. Concatenate them in uppercase into a single string with no separators. Output that string as your answer. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
RMOEUOcorrectif.format.repeat-v1anchorconf 100% · 817ms · $0.001 · 244 tok
model answer:
ECHO_ECHO_ECHO_ECHO_ECHO_ECHO_ECHO_ECHOcorrectif.format.repeat-v1anchorconf 100% · 1.1s · $0.001 · 359 tok
model answer:
ZEPHYR/ZEPHYR/ZEPHYR/ZEPHYR/ZEPHYR/ZEPHYRcorrectif.format.acronym-v1anchorconf 100% · 1.3s · $0.000 · 156 tok
model answer:
ZDFQknowledge 30/30 correct
correctknowledge.fr.factbank-v2conf 100% · 517ms · $0.001 · 303 tok
question
Identify the element whose symbol is Hg. Answer with the name only — no explanation. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Mercurycorrectknowledge.fr.factbank-v2conf 100% · 3.2s · $0.000 · 100 tok
question
What is the element whose symbol is Sb? Answer with the name only — no explanation. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Antimonycorrectknowledge.fr.factbank-v2conf 100% · 687ms · $0.001 · 490 tok
question
Identify the capital of Brazil. Answer with the name only — no explanation. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Brasíliacorrectknowledge.fr.factbank-v2conf 100% · 754ms · $0.000 · 119 tok
question
Name the Swiss capital (de facto). Answer with the name only — no explanation. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Berncorrectknowledge.fr.factbank-v2conf 100% · 602ms · $0.002 · 822 tok
question
Name the element whose symbol is Sn. Answer with the name only — no explanation. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Tincorrectknowledge.fr.factbank-v2conf 100% · 1.0s · $0.001 · 451 tok
question
Identify the Swiss capital (de facto). Answer with the name only — no explanation. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Berncorrectknowledge.fr.factbank-v2conf 100% · 989ms · $0.001 · 734 tok
question
Identify the capital of Switzerland. Answer with the name only — no explanation. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Berncorrectknowledge.fr.factbank-v2conf 100% · 1.3s · $0.001 · 390 tok
question
Name the element whose symbol is Sn. Answer with the name only — no explanation. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Tincorrectknowledge.fr.factbank-v2conf 100% · 1.5s · $0.000 · 136 tok
question
Identify the capital of Turkey. Answer with the name only — no explanation. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Ankaracorrectknowledge.fr.factbank-v2conf 100% · 1.2s · $0.001 · 251 tok
question
Identify the chemical element with symbol Sb. Answer with the name only — no explanation. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Antimonycorrectknowledge.fr.factbank-v2conf 100% · 700ms · $0.001 · 573 tok
question
Name the Burmese capital city. Answer with the name only — no explanation. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Naypyidawcorrectknowledge.fr.factbank-v2conf 100% · 842ms · $0.000 · 125 tok
question
Identify the Brazilian capital city. Answer with the name only — no explanation. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Brasíliacorrectknowledge.fr.factbank-v2conf 100% · 938ms · $0.001 · 360 tok
question
What is the chemical element with symbol K? Answer with the name only — no explanation. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Potassiumcorrectknowledge.fr.factbank-v2conf 100% · 1.0s · $0.001 · 478 tok
question
Identify the chemical element with symbol Hg. Answer with the name only — no explanation. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Mercurycorrectknowledge.fr.factbank-v2conf 100% · 802ms · $0.001 · 413 tok
question
What is the Brazilian capital city? Answer with the name only — no explanation. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Brasíliacorrectknowledge.fr.factbank-v2conf 100% · 507ms · $0.001 · 286 tok
question
What is the chemical element with symbol W? Answer with the name only — no explanation. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Tungstencorrectknowledge.fr.factbank-v2conf 100% · 1.1s · $0.002 · 1004 tok
question
What is the capital of Australia? Answer with the name only — no explanation. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Canberracorrectknowledge.fr.factbank-v2conf 100% · 1.1s · $0.001 · 678 tok
question
What is the capital of Turkey? Answer with the name only — no explanation. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Ankaracorrectknowledge.fr.factbank-v2conf 100% · 1.1s · $0.000 · 92 tok
question
Identify the element whose symbol is W. Answer with the name only — no explanation. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Tungstencorrectknowledge.fr.factbank-v2conf 100% · 903ms · $0.000 · 228 tok
question
What is the writer of the novel "The Master and Margarita"? Answer with the name only — no explanation. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Mikhail Bulgakovcorrectknowledge.fr.factbank-v2conf 100% · 707ms · $0.001 · 364 tok
question
Name the Canadian capital city. Answer with the name only — no explanation. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Ottawacorrectknowledge.fr.factbank-v2conf 100% · 1.6s · $0.001 · 240 tok
question
Identify the chemical element with symbol W. Answer with the name only — no explanation. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Tungstencorrectknowledge.fr.factbank-v2conf 100% · 989ms · $0.001 · 302 tok
question
What is the chemical element with symbol Sb? Answer with the name only — no explanation. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Antimonycorrectknowledge.fr.factbank-v2conf 100% · 1.6s · $0.001 · 563 tok
question
What is the Canadian capital city? Answer with the name only — no explanation. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Ottawacorrectknowledge.fr.factbank-v2conf 100% · 818ms · $0.002 · 797 tok
question
What is the Nigerian capital city? Answer with the name only — no explanation. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Abujacorrectknowledge.fr.factbank-v2conf 100% · 506ms · $0.000 · 123 tok
question
What is the element whose symbol is Sb? Answer with the name only — no explanation. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Antimonycorrectknowledge.fr.factbank-v2anchorconf 100% · 710ms · $0.001 · 466 tok
model answer:
Mercurycorrectknowledge.fr.factbank-v2anchorconf 100% · 1.6s · $0.002 · 881 tok
model answer:
Antimonycorrectknowledge.fr.factbank-v2anchorconf 100% · 1.2s · $0.001 · 257 tok
model answer:
Tungstencorrectknowledge.fr.factbank-v2anchorconf 100% · 1.6s · $0.002 · 742 tok
model answer:
Leadmath 30/30 correct
correctmath.counterfactual.base-v1conf 100% · 861ms · $0.003 · 1637 tok
question
Work strictly in base 13. Multiply the base-13 numbers 61 and 4A. Give the result IN BASE 13 (digits beyond 9 are A, B, C). End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
22CAcorrectmath.chained.pipeline-v1conf 100% · 2.2s · $0.001 · 677 tok
question
Solve the following linked steps; each step uses the previous result. Step 1: P = 87 × 34. Step 2: Q = P × 4 − 663. Step 3: divide Q by 3: let q be the integer quotient and r the remainder. The final answer is q + r. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
3723correctmath.percent.chain-v2conf 100% · 829ms · $0.002 · 953 tok
question
An inventory starts at 17000 units. Each pallet weighs about 20 grams more when wet. In the first month the inventory grows by 12%. Each pallet weighs about 96 grams more when wet. The next month it shrinks by 16%, and the month after it grows by 25%. How many units remain (exact value, round to 2 decimals only if needed)? End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
19992correctmath.algebra.system-v2conf 100% · 3.1s · $0.001 · 725 tok
question
Solve the system, then answer the derived question. 5x + 7y = -293 4x − 6y = 44 What is the value of 3x − 5y? End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
45correctmath.arith.chain-v2conf 100% · 1.0s · $0.002 · 1061 tok
question
Compute the value of the following expression. (((55 × 43 − 367) × 5 + 3165) − 32 × 18) × 4 End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
50316correctmath.counterfactual.base-v1conf 100% · 1.1s · $0.003 · 1524 tok
question
Work strictly in base 13. Multiply the base-13 numbers 54 and 57. Give the result IN BASE 13 (digits beyond 9 are A, B, C). End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
2352correctmath.chained.pipeline-v1conf 100% · 563ms · $0.001 · 635 tok
question
Solve the following linked steps; each step uses the previous result. Step 1: P = 36 × 81. Step 2: Q = P × 4 − 577. Step 3: divide Q by 7: let q be the integer quotient and r the remainder. The final answer is q + r. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
1589correctmath.percent.chain-v2conf 100% · 1.6s · $0.002 · 1020 tok
question
An inventory starts at 9000 units. The warehouse was painted 42 years ago. In the first month the inventory grows by 8%. The delivery van has a 95-liter fuel tank. The next month it shrinks by 20%, and the month after it grows by 8%. How many units remain (exact value, round to 2 decimals only if needed)? End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
8398.08correctmath.arith.chain-v2conf 100% · 1.4s · $0.002 · 1168 tok
question
Work out the exact value of this expression. (((64 × 43 − 841) × 3 + 7189) − 46 × 82) × 6 End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
54900correctmath.algebra.system-v2conf 100% · 1.2s · $0.001 · 445 tok
question
Solve the system, then answer the derived question. 3x + 8y = -102 2x − 6y = 170 What is the value of 2x − 6y? End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
170correctmath.chained.pipeline-v1conf 100% · 1.9s · $0.001 · 699 tok
question
Solve the following linked steps; each step uses the previous result. Step 1: P = 81 × 12. Step 2: Q = P × 6 − 763. Step 3: divide Q by 9: let q be the integer quotient and r the remainder. The final answer is q + r. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
565correctmath.counterfactual.base-v1conf 100% · 818ms · $0.002 · 1141 tok
question
Work strictly in base 9. Add the base-9 numbers 2253 and 3600. Give the result IN BASE 9. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
5853correctmath.percent.chain-v2conf 100% · 543ms · $0.004 · 1764 tok
question
An inventory starts at 64000 units. Each pallet weighs about 168 grams more when wet. In the first month the inventory grows by 29%. The delivery van has a 172-liter fuel tank. The next month it shrinks by 17%, and the month after it grows by 41%. How many units remain (exact value, round to 2 decimals only if needed)? End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
96619.97correctmath.arith.chain-v2conf 100% · 1.1s · $0.004 · 1878 tok
question
Calculate the following. Show your reasoning, then answer. (((71 × 96 − 181) × 5 + 3214) − 85 × 32) × 6 End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
202014correctmath.algebra.system-v2conf 100% · 1.3s · $0.001 · 712 tok
question
Solve the system, then answer the derived question. 4x + 8y = 44 6x − 7y = 332 What is the value of 5x − 6y? End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
279correctmath.counterfactual.base-v1conf 100% · 1.6s · $0.002 · 1106 tok
question
Work strictly in base 9. Multiply the base-9 numbers 74 and 61. Give the result IN BASE 9. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
5044correctmath.chained.pipeline-v1conf 100% · 1.3s · $0.001 · 535 tok
question
Solve the following linked steps; each step uses the previous result. Step 1: P = 38 × 16. Step 2: Q = P × 3 − 455. Step 3: divide Q by 6: let q be the integer quotient and r the remainder. The final answer is q + r. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
229correctmath.percent.chain-v2conf 100% · 884ms · $0.002 · 749 tok
question
An inventory starts at 2000 units. A rival firm shipped 26 unrelated parcels the same week. In the first month the inventory grows by 29%. The delivery van has a 73-liter fuel tank. The next month it shrinks by 21%, and the month after it grows by 35%. How many units remain (exact value, round to 2 decimals only if needed)? End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
2751.57correctmath.arith.chain-v2conf 100% · 960ms · $0.003 · 1429 tok
question
Evaluate the expression below and give the result. (((76 × 42 − 405) × 6 + 3180) − 74 × 25) × 2 End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
36104correctmath.algebra.system-v2conf 100% · 1.1s · $0.001 · 537 tok
question
Solve the system, then answer the derived question. 8x + 3y = -162 8x − 4y = -400 What is the value of 5x − 5y? End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
-335correctmath.chained.pipeline-v1conf 100% · 569ms · $0.001 · 501 tok
question
Solve the following linked steps; each step uses the previous result. Step 1: P = 32 × 22. Step 2: Q = P × 8 − 173. Step 3: divide Q by 4: let q be the integer quotient and r the remainder. The final answer is q + r. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
1367correctmath.counterfactual.base-v1conf 100% · 567ms · $0.003 · 1297 tok
question
Work strictly in base 8. Add the base-8 numbers 5243 and 2613. Give the result IN BASE 8. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
10056correctmath.percent.chain-v2conf 100% · 1.6s · $0.002 · 841 tok
question
An inventory starts at 36000 units. The warehouse was painted 113 years ago. In the first month the inventory grows by 45%. A rival firm shipped 92 unrelated parcels the same week. The next month it shrinks by 12%, and the month after it grows by 22%. How many units remain (exact value, round to 2 decimals only if needed)? End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
56041.92correctmath.algebra.system-v2conf 100% · 1.7s · $0.002 · 787 tok
question
Solve the system, then answer the derived question. 7x + 3y = 137 2x − 2y = 22 What is the value of 5x − 2y? End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
73correctmath.arith.chain-v2conf 100% · 1.1s · $0.002 · 1171 tok
question
Work out the exact value of this expression. (((60 × 54 − 605) × 8 + 6709) − 19 × 56) × 4 End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
106900correctmath.chained.pipeline-v1conf 100% · 2.6s · $0.002 · 1187 tok
question
Solve the following linked steps; each step uses the previous result. Step 1: P = 69 × 68. Step 2: Q = P × 9 − 566. Step 3: divide Q by 6: let q be the integer quotient and r the remainder. The final answer is q + r. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
6947correctmath.counterfactual.base-v1anchorconf 100% · 528ms · $0.002 · 1167 tok
model answer:
11236correctmath.percent.chain-v2anchorconf 100% · 1.1s · $0.003 · 1572 tok
model answer:
61896.52correctmath.arith.chain-v2anchorconf 100% · 1.1s · $0.003 · 1290 tok
model answer:
108153correctmath.algebra.system-v2anchorconf 100% · 1.1s · $0.002 · 741 tok
model answer:
87multilingual 30/30 correct
correctmultilingual.numword-v2conf 100% · 519ms · $0.001 · 602 tok
question
Compute 422 + 193, then write the result out in Spanish number words (lowercase). Answer with the Spanish words only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
seiscientos quincecorrectmultilingual.wordnum-v1conf 100% · 825ms · $0.001 · 295 tok
question
A number is written in French: « cinq cent quatre-vingt-trois ». Another is written in Spanish: « ochocientos veintisiete ». Compute (French number) + (Spanish number). Answer with digits only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
1410correctmultilingual.numword-v2conf 100% · 1.8s · $0.001 · 557 tok
question
Compute 131 + 153, then write the result out in French number words (lowercase). Answer with the French words only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
deux cent quatre-vingt-quatrecorrectmultilingual.wordnum-v1conf 100% · 510ms · $0.001 · 332 tok
question
A number is written in French: « cent douze ». Another is written in Spanish: « trescientos treinta y seis ». Compute (French number) + (Spanish number). Answer with digits only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
448correctmultilingual.numword-v2conf 100% · 625ms · $0.001 · 524 tok
question
Compute 148 + 242, then write the result out in French number words (lowercase). Answer with the French words only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
trois cent quatre-vingt-dixcorrectmultilingual.wordnum-v1conf 100% · 1.3s · $0.001 · 384 tok
question
A number is written in French: « six cent quatre-vingt-quatorze ». Another is written in Spanish: « quinientos noventa y nueve ». Compute (French number) − (Spanish number). Answer with digits only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
95correctmultilingual.wordnum-v1conf 100% · 745ms · $0.001 · 321 tok
question
A number is written in French: « quatre cent quatre-vingt-quatorze ». Another is written in Spanish: « ciento noventa y uno ». Compute (French number) + (Spanish number). Answer with digits only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
685correctmultilingual.numword-v2conf 100% · 683ms · $0.001 · 319 tok
question
Compute 59 + 225, then write the result out in Spanish number words (lowercase). Answer with the Spanish words only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
doscientos ochenta y cuatrocorrectmultilingual.wordnum-v1conf 100% · 1.1s · $0.001 · 300 tok
question
A number is written in French: « trois cent cinq ». Another is written in Spanish: « setecientos cuarenta y dos ». Compute (French number) + (Spanish number). Answer with digits only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
1047correctmultilingual.numword-v2conf 100% · 1.1s · $0.001 · 326 tok
question
Compute 358 + 170, then write the result out in Spanish number words (lowercase). Answer with the Spanish words only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
quinientos veintiochocorrectmultilingual.numword-v2conf 100% · 1.0s · $0.002 · 826 tok
question
Compute 303 + 314, then write the result out in Spanish number words (lowercase). Answer with the Spanish words only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
seiscientos diecisietecorrectmultilingual.wordnum-v1conf 100% · 1.4s · $0.001 · 456 tok
question
A number is written in French: « neuf cent dix ». Another is written in Spanish: « novecientos cuatro ». Compute (French number) + (Spanish number). Answer with digits only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
1814correctmultilingual.numword-v2conf 100% · 1.2s · $0.001 · 643 tok
question
Compute 438 + 82, then write the result out in Spanish number words (lowercase). Answer with the Spanish words only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
quinientos veintecorrectmultilingual.wordnum-v1conf 100% · 720ms · $0.001 · 303 tok
question
A number is written in French: « quatre cent soixante-six ». Another is written in Spanish: « sesenta y siete ». Compute (French number) − (Spanish number). Answer with digits only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
399correctmultilingual.wordnum-v1conf 100% · 623ms · $0.000 · 209 tok
question
A number is written in French: « cinq cent quatre-vingt-six ». Another is written in Spanish: « cincuenta y nueve ». Compute (French number) + (Spanish number). Answer with digits only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
645correctmultilingual.numword-v2conf 100% · 504ms · $0.001 · 328 tok
question
Compute 350 + 267, then write the result out in French number words (lowercase). Answer with the French words only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
six cent dix-septcorrectmultilingual.numword-v2conf 100% · 1.1s · $0.001 · 679 tok
question
Compute 155 + 333, then write the result out in French number words (lowercase). Answer with the French words only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
quatre cent quatre-vingt-huitcorrectmultilingual.wordnum-v1conf 100% · 1.2s · $0.001 · 424 tok
question
A number is written in French: « trois cent cinquante-neuf ». Another is written in Spanish: « ochocientos ochenta y nueve ». Compute (French number) + (Spanish number). Answer with digits only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
1248correctmultilingual.wordnum-v1conf 100% · 1.5s · $0.001 · 251 tok
question
A number is written in French: « deux cent quarante-sept ». Another is written in Spanish: « doscientos treinta y cuatro ». Compute (French number) + (Spanish number). Answer with digits only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
481correctmultilingual.numword-v2conf 100% · 856ms · $0.001 · 401 tok
question
Compute 458 + 400, then write the result out in Spanish number words (lowercase). Answer with the Spanish words only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
ochocientos cincuenta y ochocorrectmultilingual.wordnum-v1conf 100% · 774ms · $0.001 · 239 tok
question
A number is written in French: « neuf cent quatre-vingt-quatre ». Another is written in Spanish: « trescientos treinta y cinco ». Compute (French number) + (Spanish number). Answer with digits only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
1319correctmultilingual.numword-v2conf 100% · 880ms · $0.001 · 593 tok
question
Compute 81 + 188, then write the result out in French number words (lowercase). Answer with the French words only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
deux cent soixante-neufcorrectmultilingual.numword-v2conf 100% · 1.0s · $0.002 · 973 tok
question
Compute 275 + 281, then write the result out in Spanish number words (lowercase). Answer with the Spanish words only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
quinientos cincuenta y seiscorrectmultilingual.wordnum-v1conf 100% · 1.1s · $0.001 · 336 tok
question
A number is written in French: « cinq cent vingt-sept ». Another is written in Spanish: « quinientos noventa y ocho ». Compute (French number) − (Spanish number). Answer with digits only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
-71correctmultilingual.wordnum-v1conf 100% · 496ms · $0.001 · 382 tok
question
A number is written in French: « six cent vingt-trois ». Another is written in Spanish: « ochocientos sesenta y cinco ». Compute (French number) − (Spanish number). Answer with digits only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
-242correctmultilingual.numword-v2conf 100% · 1.7s · $0.001 · 348 tok
question
Compute 195 + 262, then write the result out in Spanish number words (lowercase). Answer with the Spanish words only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
cuatrocientos cincuenta y sietecorrectmultilingual.wordnum-v1anchorconf 100% · 1.9s · $0.001 · 302 tok
model answer:
150correctmultilingual.numword-v2anchorconf 100% · 771ms · $0.001 · 419 tok
model answer:
huit cent soixante-dix-neufcorrectmultilingual.numword-v2anchorconf 100% · 1.3s · $0.001 · 418 tok
model answer:
seiscientos ochocorrectmultilingual.wordnum-v1anchorconf 100% · 1.6s · $0.001 · 358 tok
model answer:
762reasoning 30/30 correct
correctreasoning.deduction.order-v2conf 100% · 715ms · $0.003 · 1702 tok
question
Seven people are ranked by who is older (rank 1 = oldest). Sami is older than Kira. Hana is faster than everyone here, but Hana is not being ranked. Mona is older than Kira. Priya is older than Bruno. Bruno is older than Sami. Sami is older than Kira. Dara is older than Priya. Goran is older than Dara. Dara is older than Bruno. Sami is older than Mona. Who is fourth (rank 4)? Answer with the name only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Brunocorrectreasoning.deduction.position-v1conf 100% · 1.1s · $0.001 · 264 tok
question
Four people stand in a queue (number 1 is the front). Sami is number 2 in the queue. Tessa is directly ahead of Sami. Kira is directly ahead of Alice. Who is number 2? Answer with the name only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Samicorrectreasoning.deduction.order-v2conf 100% · 565ms · $0.002 · 1179 tok
question
Seven people are ranked by who is taller (rank 1 = tallest). Dara is taller than Tessa. Nadir is taller than Bruno. Tessa is taller than Sami. Sami is taller than Alice. Bruno is taller than Sami. Bruno is taller than Rosa. Dara is taller than Sami. Bruno is taller than Alice. Hana is heavier than everyone here, but Hana is not being ranked. Rosa is taller than Dara. Who is second (rank 2)? Answer with the name only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Brunocorrectreasoning.deduction.position-v1conf 100% · 853ms · $0.001 · 366 tok
question
Four people stand in a queue (number 1 is the front). Jonas is directly ahead of Bruno. Mona is directly ahead of Alice. Bruno is number 2 in the queue. Who is number 2? Answer with the name only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Brunocorrectreasoning.deduction.order-v2conf 100% · 1.9s · $0.004 · 1777 tok
question
Seven people are ranked by who is older (rank 1 = oldest). Jonas is older than Ola. Chen is older than Ola. Quinn is taller than everyone here, but Quinn is not being ranked. Jonas is older than Mona. Mona is older than Chen. Goran is older than Kira. Mona is older than Goran. Alice is older than Jonas. Goran is older than Chen. Kira is older than Chen. Who is third (rank 3)? Answer with the name only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Monacorrectreasoning.deduction.position-v1conf 100% · 803ms · $0.001 · 294 tok
question
Four people stand in a queue (number 1 is the front). Dara is number 1 in the queue. Kira is directly ahead of Goran. Nadir is directly ahead of Kira. Who is number 4? Answer with the name only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Gorancorrectreasoning.deduction.order-v2conf 100% · 1.6s · $0.003 · 1237 tok
question
Seven people are ranked by who is taller (rank 1 = tallest). Emil is taller than Quinn. Chen is taller than Quinn. Chen is taller than Priya. Tessa is taller than Chen. Jonas is taller than Quinn. Jonas is taller than Bruno. Ines is faster than everyone here, but Ines is not being ranked. Bruno is taller than Priya. Bruno is taller than Tessa. Priya is taller than Emil. Who is fourth (rank 4)? Answer with the name only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Chencorrectreasoning.deduction.position-v1conf 100% · 1.3s · $0.000 · 220 tok
question
Four people stand in a queue (number 1 is the front). Chen is number 1 in the queue. Goran is directly ahead of Ola. Ola is directly ahead of Dara. Who is number 1? Answer with the name only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Chencorrectreasoning.deduction.order-v2conf 100% · 1.1s · $0.002 · 915 tok
question
Seven people are ranked by who is faster (rank 1 = fastest). Hana is faster than Nadir. Bruno is faster than Dara. Tessa is faster than Ines. Tessa is faster than Hana. Ines is faster than Bruno. Ines is faster than Dara. Nadir is faster than Ines. Hana is faster than Dara. Dara is faster than Emil. Priya is older than everyone here, but Priya is not being ranked. Who is sixth (rank 6)? Answer with the name only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Daracorrectreasoning.deduction.position-v1conf 100% · 656ms · $0.000 · 196 tok
question
Four people stand in a queue (number 1 is the front). Tessa is directly ahead of Ines. Chen is number 1 in the queue. Hana is directly ahead of Tessa. Who is number 1? Answer with the name only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Chencorrectreasoning.deduction.order-v2conf 100% · 1.5s · $0.002 · 1081 tok
question
Seven people are ranked by who is faster (rank 1 = fastest). Dara is faster than Priya. Priya is faster than Bruno. Ines is faster than Farah. Quinn is faster than Dara. Quinn is faster than Sami. Farah is faster than Quinn. Dara is faster than Bruno. Sami is faster than Dara. Rosa is older than everyone here, but Rosa is not being ranked. Dara is faster than Bruno. Who is fourth (rank 4)? Answer with the name only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Samicorrectreasoning.deduction.position-v1conf 100% · 1.1s · $0.001 · 240 tok
question
Four people stand in a queue (number 1 is the front). Jonas is directly ahead of Mona. Mona is directly ahead of Tessa. Alice is number 4 in the queue. Tessa is directly ahead of Alice. Who is number 4? Answer with the name only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Alicecorrectreasoning.deduction.order-v2conf 100% · 884ms · $0.002 · 1208 tok
question
Seven people are ranked by who is faster (rank 1 = fastest). Dara is faster than Priya. Chen is faster than Dara. Dara is faster than Kira. Priya is faster than Tessa. Dara is faster than Priya. Kira is faster than Nadir. Alice is taller than everyone here, but Alice is not being ranked. Jonas is faster than Priya. Tessa is faster than Kira. Dara is faster than Jonas. Who is second (rank 2)? Answer with the name only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Daracorrectreasoning.deduction.position-v1conf 100% · 572ms · $0.001 · 278 tok
question
Four people stand in a queue (number 1 is the front). Bruno is number 1 in the queue. Mona is directly ahead of Emil. Tessa is directly ahead of Mona. Who is number 1? Answer with the name only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Brunocorrectreasoning.deduction.position-v1conf 100% · 727ms · $0.001 · 603 tok
question
Four people stand in a queue (number 1 is the front). Rosa is directly ahead of Priya. Chen is number 3 in the queue. Priya is directly ahead of Chen. Who is number 2? Answer with the name only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Priyacorrectreasoning.deduction.order-v2conf 100% · 684ms · $0.002 · 1147 tok
question
Seven people are ranked by who is taller (rank 1 = tallest). Priya is taller than Nadir. Tessa is taller than Bruno. Nadir is taller than Jonas. Nadir is taller than Farah. Jonas is taller than Tessa. Alice is faster than everyone here, but Alice is not being ranked. Bruno is taller than Hana. Jonas is taller than Bruno. Nadir is taller than Farah. Hana is taller than Farah. Who is second (rank 2)? Answer with the name only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Nadircorrectreasoning.deduction.position-v1conf 100% · 798ms · $0.004 · 2038 tok
question
Four people stand in a queue (number 1 is the front). Hana is number 2 in the queue. Jonas is directly ahead of Nadir. Rosa is directly ahead of Hana. Who is number 2? Answer with the name only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Hanacorrectreasoning.deduction.order-v2conf 100% · 909ms · $0.004 · 1913 tok
question
Seven people are ranked by who is taller (rank 1 = tallest). Jonas is taller than Ola. Jonas is taller than Quinn. Hana is taller than Ola. Liam is taller than Ola. Sami is taller than Jonas. Hana is taller than Liam. Rosa is taller than Hana. Sami is taller than Hana. Bruno is faster than everyone here, but Bruno is not being ranked. Quinn is taller than Rosa. Who is sixth (rank 6)? Answer with the name only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Liamcorrectreasoning.deduction.order-v2conf 100% · 1.2s · $0.002 · 1076 tok
question
Seven people are ranked by who is faster (rank 1 = fastest). Sami is faster than Chen. Nadir is faster than Bruno. Farah is faster than Sami. Tessa is faster than Farah. Farah is faster than Chen. Sami is faster than Liam. Nadir is faster than Tessa. Quinn is taller than everyone here, but Quinn is not being ranked. Liam is faster than Chen. Bruno is faster than Tessa. Who is second (rank 2)? Answer with the name only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Brunocorrectreasoning.deduction.position-v1conf 100% · 1.9s · $0.001 · 323 tok
question
Four people stand in a queue (number 1 is the front). Ines is directly ahead of Bruno. Sami is directly ahead of Jonas. Jonas is number 2 in the queue. Who is number 4? Answer with the name only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Brunocorrectreasoning.deduction.order-v2conf 100% · 1.1s · $0.003 · 1268 tok
question
Seven people are ranked by who is heavier (rank 1 = heaviest). Jonas is heavier than Emil. Jonas is heavier than Ines. Mona is heavier than Ines. Ola is heavier than Mona. Hana is heavier than Tessa. Tessa is heavier than Ines. Mona is heavier than Hana. Dara is faster than everyone here, but Dara is not being ranked. Mona is heavier than Ines. Emil is heavier than Ola. Who is second (rank 2)? Answer with the name only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Emilcorrectreasoning.deduction.position-v1conf 100% · 1.0s · $0.002 · 775 tok
question
Four people stand in a queue (number 1 is the front). Jonas is number 3 in the queue. Chen is directly ahead of Jonas. Ines is directly ahead of Chen. Who is number 3? Answer with the name only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Jonascorrectreasoning.deduction.order-v2conf 100% · 524ms · $0.003 · 1345 tok
question
Seven people are ranked by who is taller (rank 1 = tallest). Hana is taller than Sami. Liam is taller than Bruno. Sami is taller than Priya. Hana is taller than Tessa. Tessa is taller than Liam. Liam is taller than Priya. Quinn is faster than everyone here, but Quinn is not being ranked. Alice is taller than Hana. Liam is taller than Sami. Bruno is taller than Sami. Who is fifth (rank 5)? Answer with the name only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Brunocorrectreasoning.deduction.position-v1conf 100% · 954ms · $0.001 · 652 tok
question
Four people stand in a queue (number 1 is the front). Quinn is directly ahead of Bruno. Dara is number 4 in the queue. Bruno is directly ahead of Dara. Nadir is directly ahead of Quinn. Who is number 3? Answer with the name only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Brunocorrectreasoning.deduction.order-v2conf 100% · 528ms · $0.002 · 1209 tok
question
Seven people are ranked by who is heavier (rank 1 = heaviest). Chen is heavier than Bruno. Ola is heavier than Bruno. Bruno is heavier than Dara. Sami is heavier than Mona. Sami is heavier than Dara. Farah is faster than everyone here, but Farah is not being ranked. Bruno is heavier than Sami. Dara is heavier than Mona. Ola is heavier than Chen. Goran is heavier than Ola. Who is fifth (rank 5)? Answer with the name only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Samicorrectreasoning.deduction.position-v1conf 100% · 512ms · $0.000 · 177 tok
question
Four people stand in a queue (number 1 is the front). Bruno is directly ahead of Chen. Alice is number 1 in the queue. Kira is directly ahead of Bruno. Who is number 1? Answer with the name only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
Alicecorrectreasoning.deduction.position-v1anchorconf 100% · 1.0s · $0.001 · 427 tok
model answer:
Quinncorrectreasoning.deduction.order-v2anchorconf 100% · 1.5s · $0.004 · 1838 tok
model answer:
Monacorrectreasoning.deduction.order-v2anchorconf 100% · 1.3s · $0.002 · 1184 tok
model answer:
Quinncorrectreasoning.deduction.position-v1anchorconf 100% · 1.8s · $0.001 · 630 tok
model answer:
Farahterminal 30/30 correct
correctterminal.fs.tree-v1conf 100% · 1.1s · $0.004 · 2046 tok
question
A POSIX shell session starts in `/proj`. The tree initially contains these FILES (directories exist as implied, plus empty dirs `/proj/conf`, `/proj/docs`, `/proj/logs`): ``` /proj/conf/notes.txt /proj/conf/util.txt /proj/draft.md /proj/logs/report.md /proj/main.md ``` These commands run in order (all succeed; `mv x dir/` moves into the directory; paths are relative to the CURRENT working directory, which `cd` changes): ```sh rm conf/notes.txt mv main.md index-1.md cd docs mv ../../proj/logs/report.md ../../proj/ cd ../../proj/conf touch ../../proj/main-3.log mkdir -p ../../proj/docs/conf-2 rm ../../proj/report.md rm util.txt cd ../../proj ``` List every file (absolute paths) that exists afterwards, one per line, sorted in byte order (C locale). Do not list directories. Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line: CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
/proj/draft.md
/proj/index-1.md
/proj/main-3.logcorrectterminal.exit.chain-v1conf 100% · 779ms · $0.001 · 370 tok
question
A POSIX shell session in a directory containing ONLY these files: ``` app.txt notes.txt ``` `notes.txt` contains exactly the words: basil, dune (one per line). No other files exist. These statements run in order: ```sh grep -q amber notes.txt && echo A || echo B test -f app.txt && echo C || echo D test -f app.txt && echo E || echo F test -f app.txt && echo Z ``` Predict the terminal output: every line printed, in order, then a final line `exit:<N>` where N is the exit status of the LAST statement. Remember: `A && B || C` runs C whenever A fails (it is not a strict if/else); `test -f` succeeds only if the file exists; `grep -q` succeeds only if the word is present. Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line: CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
B
C
E
Z
exit:0correctterminal.exit.chain-v1conf 100% · 1.2s · $0.002 · 842 tok
question
A POSIX shell session in a directory containing ONLY these files: ``` app.txt data.txt notes.txt ``` `notes.txt` contains exactly the words: basil, coral (one per line). No other files exist. These statements run in order: ```sh grep -q coral notes.txt && echo A || echo B test -f data.txt && echo C || echo D test -f app.txt && echo E || echo F grep -q basil notes.txt && echo G || echo H test -f ghost.txt && echo Z ``` Predict the terminal output: every line printed, in order, then a final line `exit:<N>` where N is the exit status of the LAST statement. Remember: `A && B || C` runs C whenever A fails (it is not a strict if/else); `test -f` succeeds only if the file exists; `grep -q` succeeds only if the word is present. Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line: CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
A
C
E
G
exit:1correctterminal.pipeline.predict-v1conf 100% · 587ms · $0.001 · 604 tok
question
A POSIX shell session (LC_ALL=C). The file `people.csv` contains exactly these lines (columns: name,dept,units,score):
```
ivy,hr,115,58
oli,legal,5,19
ana,hr,49,39
jon,sales,28,85
cy,legal,74,51
dev,ops,98,92
pam,legal,70,71
max,hr,28,81
gus,ops,36,58
ned,eng,53,17
fay,eng,115,21
```
What is the EXACT stdout of this command?
```sh
grep -F ',hr,' people.csv | awk -F, '$4 > 59 { n += 1 } END { print n }'
```
Notes: plain `sort` compares bytes (so "100" sorts before "9"); `sort -k3,3n` compares field 3 numerically.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
1correctterminal.pipeline.predict-v1conf 100% · 810ms · $0.003 · 1531 tok
question
A POSIX shell session (LC_ALL=C). The file `people.csv` contains exactly these lines (columns: name,dept,units,score): ``` jon,legal,19,64 hal,sales,44,20 eli,legal,70,93 gus,hr,115,15 max,eng,46,77 oli,sales,30,69 ana,hr,117,97 lou,hr,94,22 cy,eng,7,60 ivy,eng,46,39 ``` What is the EXACT stdout of this command? ```sh grep -F ',sales,' people.csv | cut -d, -f1,3 | sort | head -n 3 ``` Notes: plain `sort` compares bytes (so "100" sorts before "9"); `sort -k3,3n` compares field 3 numerically. Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line: CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
hal,44
oli,30correctterminal.fs.tree-v1conf 100% · 832ms · $0.003 · 1483 tok
question
A POSIX shell session starts in `/proj`. The tree initially contains these FILES (directories exist as implied, plus empty dirs `/proj/src`, `/proj/docs`, `/proj/assets`): ``` /proj/assets/notes.txt /proj/docs/main.cfg /proj/draft.cfg /proj/src/report.log /proj/util.txt ``` These commands run in order (all succeed; `mv x dir/` moves into the directory; paths are relative to the CURRENT working directory, which `cd` changes): ```sh mkdir -p conf-9 mkdir -p docs-6 cd . mkdir -p docs/conf-8 mv docs/main.cfg docs/setup-2.txt mkdir -p assets-9 touch index-9.log ``` List every file (absolute paths) that exists afterwards, one per line, sorted in byte order (C locale). Do not list directories. Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line: CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
/proj/assets/notes.txt
/proj/docs/setup-2.txt
/proj/draft.cfg
/proj/index-9.log
/proj/src/report.log
/proj/util.txtcorrectterminal.exit.chain-v1conf 100% · 1.2s · $0.002 · 1066 tok
question
A POSIX shell session in a directory containing ONLY these files: ``` app.txt data.txt notes.txt ``` `notes.txt` contains exactly the words: dune, basil (one per line). No other files exist. These statements run in order: ```sh test -f app.txt && echo A || echo B true && echo C || echo D true && echo E || echo F grep -q coral notes.txt && echo G || echo H test -f app.txt && echo Z ``` Predict the terminal output: every line printed, in order, then a final line `exit:<N>` where N is the exit status of the LAST statement. Remember: `A && B || C` runs C whenever A fails (it is not a strict if/else); `test -f` succeeds only if the file exists; `grep -q` succeeds only if the word is present. Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line: CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
A
C
E
H
Z
exit:0correctterminal.fs.tree-v1conf 100% · 3.3s · $0.002 · 1039 tok
question
A POSIX shell session starts in `/proj`. The tree initially contains these FILES (directories exist as implied, plus empty dirs `/proj/assets`, `/proj/build`, `/proj/src`): ``` /proj/assets/index.txt /proj/build/setup.log /proj/build/util.md /proj/main.cfg /proj/notes.cfg ``` These commands run in order (all succeed; `mv x dir/` moves into the directory; paths are relative to the CURRENT working directory, which `cd` changes): ```sh cp assets/index.txt src/ mv src/index.txt src/setup-2.md rm assets/index.txt touch assets/setup-8.cfg touch assets/todo-3.cfg mkdir -p assets/logs-2 cd src ``` List every file (absolute paths) that exists afterwards, one per line, sorted in byte order (C locale). Do not list directories. Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line: CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
/proj/assets/setup-8.cfg
/proj/assets/todo-3.cfg
/proj/build/setup.log
/proj/build/util.md
/proj/main.cfg
/proj/notes.cfg
/proj/src/setup-2.mdcorrectterminal.fs.tree-v1conf 100% · 1.0s · $0.005 · 2260 tok
question
A POSIX shell session starts in `/proj`. The tree initially contains these FILES (directories exist as implied, plus empty dirs `/proj/build`, `/proj/logs`, `/proj/conf`): ``` /proj/build/index.cfg /proj/conf/main.md /proj/logs/util.md /proj/report.cfg /proj/todo.md ``` These commands run in order (all succeed; `mv x dir/` moves into the directory; paths are relative to the CURRENT working directory, which `cd` changes): ```sh mv logs/util.md logs/index-2.md cd . touch report-1.md mkdir -p conf/logs-3 cd conf/logs-3 mkdir -p ../../../proj/logs/assets-6 rm ../../../proj/logs/index-2.md mv ../../../proj/build/index.cfg ../../../proj/build/notes-2.log ``` List every file (absolute paths) that exists afterwards, one per line, sorted in byte order (C locale). Do not list directories. Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line: CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
/proj/build/notes-2.log
/proj/conf/main.md
/proj/report-1.md
/proj/report.cfg
/proj/todo.mdcorrectterminal.pipeline.predict-v1conf 100% · 1.2s · $0.002 · 1120 tok
question
A POSIX shell session (LC_ALL=C). The file `people.csv` contains exactly these lines (columns: name,dept,units,score): ``` ned,legal,108,26 hal,sales,39,32 max,hr,20,80 kim,hr,50,74 bo,hr,60,93 lou,eng,27,72 cy,sales,13,99 ivy,sales,107,78 pam,hr,115,19 oli,ops,45,27 jon,hr,14,75 eli,ops,76,43 ``` What is the EXACT stdout of this command? ```sh grep -F ',ops,' people.csv | sort -t, -k3,3n | tail -n 3 ``` Notes: plain `sort` compares bytes (so "100" sorts before "9"); `sort -k3,3n` compares field 3 numerically. Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line: CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
oli,ops,45,27
eli,ops,76,43correctterminal.exit.chain-v1conf 100% · 1.4s · $0.001 · 434 tok
question
A POSIX shell session in a directory containing ONLY these files: ``` app.txt notes.txt ``` `notes.txt` contains exactly the words: dune, basil (one per line). No other files exist. These statements run in order: ```sh test -f ghost.txt && echo A || echo B test -f app.txt && echo C || echo D test -f app.txt && echo E || echo F test -f app.txt && echo Z ``` Predict the terminal output: every line printed, in order, then a final line `exit:<N>` where N is the exit status of the LAST statement. Remember: `A && B || C` runs C whenever A fails (it is not a strict if/else); `test -f` succeeds only if the file exists; `grep -q` succeeds only if the word is present. Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line: CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
B
C
E
Z
exit:0correctterminal.fs.tree-v1conf 100% · 994ms · $0.006 · 2847 tok
question
A POSIX shell session starts in `/proj`. The tree initially contains these FILES (directories exist as implied, plus empty dirs `/proj/src`, `/proj/conf`, `/proj/docs`): ``` /proj/conf/draft.log /proj/docs/util.txt /proj/notes.log /proj/setup.cfg /proj/src/main.cfg ``` These commands run in order (all succeed; `mv x dir/` moves into the directory; paths are relative to the CURRENT working directory, which `cd` changes): ```sh rm docs/util.txt rm notes.log mkdir -p assets-4 cp src/main.cfg conf/ cp conf/draft.log docs/ cp docs/draft.log ./ mkdir -p conf/assets-4 cd . ``` List every file (absolute paths) that exists afterwards, one per line, sorted in byte order (C locale). Do not list directories. Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line: CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
/proj/conf/draft.log
/proj/conf/main.cfg
/proj/docs/draft.log
/proj/draft.log
/proj/setup.cfg
/proj/src/main.cfgcorrectterminal.pipeline.predict-v1conf 100% · 954ms · $0.002 · 886 tok
question
A POSIX shell session (LC_ALL=C). The file `people.csv` contains exactly these lines (columns: name,dept,units,score):
```
jon,sales,42,39
gus,eng,119,61
dev,eng,107,32
bo,hr,112,24
fay,ops,30,85
ana,sales,102,55
cy,eng,80,64
kim,ops,93,67
pam,sales,22,42
hal,ops,101,95
oli,hr,99,20
max,sales,109,95
lou,hr,93,13
```
What is the EXACT stdout of this command?
```sh
grep -F ',ops,' people.csv | awk -F, '{ s += $3 } END { print s }'
```
Notes: plain `sort` compares bytes (so "100" sorts before "9"); `sort -k3,3n` compares field 3 numerically.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
224correctterminal.exit.chain-v1conf 100% · 853ms · $0.002 · 754 tok
question
A POSIX shell session in a directory containing ONLY these files: ``` app.txt data.txt notes.txt ``` `notes.txt` contains exactly the words: basil, dune (one per line). No other files exist. These statements run in order: ```sh test -f app.txt && echo A || echo B test -f tmp.txt && echo C || echo D grep -q amber notes.txt && echo E || echo F test -f ghost.txt && echo Z ``` Predict the terminal output: every line printed, in order, then a final line `exit:<N>` where N is the exit status of the LAST statement. Remember: `A && B || C` runs C whenever A fails (it is not a strict if/else); `test -f` succeeds only if the file exists; `grep -q` succeeds only if the word is present. Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line: CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
A
D
F
exit:1correctterminal.pipeline.predict-v1conf 100% · 2.7s · $0.003 · 1300 tok
question
A POSIX shell session (LC_ALL=C). The file `people.csv` contains exactly these lines (columns: name,dept,units,score): ``` eli,ops,108,79 ned,legal,90,20 lou,eng,102,49 gus,eng,88,13 oli,hr,91,34 hal,legal,83,68 pam,legal,59,33 cy,hr,117,36 kim,sales,85,90 ivy,hr,108,56 ``` What is the EXACT stdout of this command? ```sh grep -F ',legal,' people.csv | cut -d, -f1,3 | sort | head -n 3 ``` Notes: plain `sort` compares bytes (so "100" sorts before "9"); `sort -k3,3n` compares field 3 numerically. Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line: CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
hal,83
ned,90
pam,59correctterminal.exit.chain-v1conf 100% · 1.2s · $0.002 · 1034 tok
question
A POSIX shell session in a directory containing ONLY these files: ``` app.txt data.txt notes.txt ``` `notes.txt` contains exactly the words: dune, amber (one per line). No other files exist. These statements run in order: ```sh grep -q dune notes.txt && echo A || echo B grep -q coral notes.txt && echo C || echo D grep -q basil notes.txt && echo E || echo F grep -q amber notes.txt && echo G || echo H test -f app.txt && echo Z ``` Predict the terminal output: every line printed, in order, then a final line `exit:<N>` where N is the exit status of the LAST statement. Remember: `A && B || C` runs C whenever A fails (it is not a strict if/else); `test -f` succeeds only if the file exists; `grep -q` succeeds only if the word is present. Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line: CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
A
D
F
G
Z
exit:0correctterminal.fs.tree-v1conf 100% · 544ms · $0.002 · 721 tok
question
A POSIX shell session starts in `/proj`. The tree initially contains these FILES (directories exist as implied, plus empty dirs `/proj/conf`, `/proj/logs`, `/proj/assets`): ``` /proj/assets/index.cfg /proj/assets/report.txt /proj/conf/notes.log /proj/main.cfg /proj/todo.md ``` These commands run in order (all succeed; `mv x dir/` moves into the directory; paths are relative to the CURRENT working directory, which `cd` changes): ```sh mkdir -p conf/assets-2 cd . cp assets/report.txt logs/ rm todo.md cp main.cfg conf/ rm main.cfg rm assets/report.txt ``` List every file (absolute paths) that exists afterwards, one per line, sorted in byte order (C locale). Do not list directories. Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line: CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
/proj/assets/index.cfg
/proj/conf/main.cfg
/proj/conf/notes.log
/proj/logs/report.txtcorrectterminal.fs.tree-v1conf 99% · 636ms · $0.005 · 2670 tok
question
A POSIX shell session starts in `/proj`. The tree initially contains these FILES (directories exist as implied, plus empty dirs `/proj/conf`, `/proj/docs`, `/proj/assets`): ``` /proj/assets/notes.log /proj/assets/report.log /proj/docs/draft.md /proj/todo.md /proj/util.txt ``` These commands run in order (all succeed; `mv x dir/` moves into the directory; paths are relative to the CURRENT working directory, which `cd` changes): ```sh rm todo.md touch todo-2.cfg cd . cp util.txt assets/ cd . touch setup-7.log rm util.txt cd assets touch ../../proj/conf/notes-4.cfg cd . cp ../../proj/docs/draft.md ./ cp ../../proj/setup-7.log ../../proj/docs/ cd . ``` List every file (absolute paths) that exists afterwards, one per line, sorted in byte order (C locale). Do not list directories. Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line: CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
/proj/assets/draft.md
/proj/assets/notes.log
/proj/assets/report.log
/proj/assets/util.txt
/proj/conf/notes-4.cfg
/proj/docs/draft.md
/proj/docs/setup-7.log
/proj/setup-7.log
/proj/todo-2.cfgcorrectterminal.pipeline.predict-v1conf 100% · 1.6s · $0.001 · 517 tok
question
A POSIX shell session (LC_ALL=C). The file `people.csv` contains exactly these lines (columns: name,dept,units,score):
```
cy,hr,65,93
lou,sales,75,90
gus,ops,54,20
dev,hr,103,15
ivy,sales,34,45
jon,eng,33,29
hal,eng,117,84
pam,eng,12,68
ned,ops,82,39
kim,sales,55,28
bo,eng,75,43
fay,ops,75,50
ana,hr,59,16
oli,ops,20,11
```
What is the EXACT stdout of this command?
```sh
grep -F ',sales,' people.csv | awk -F, '{ s += $3 } END { print s }'
```
Notes: plain `sort` compares bytes (so "100" sorts before "9"); `sort -k3,3n` compares field 3 numerically.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
164correctterminal.exit.chain-v1conf 100% · 1.2s · $0.001 · 548 tok
question
A POSIX shell session in a directory containing ONLY these files: ``` app.txt data.txt notes.txt ``` `notes.txt` contains exactly the words: amber, basil (one per line). No other files exist. These statements run in order: ```sh grep -q amber notes.txt && echo A || echo B false && echo C || echo D test -f ghost.txt && echo E || echo F test -f app.txt && echo Z ``` Predict the terminal output: every line printed, in order, then a final line `exit:<N>` where N is the exit status of the LAST statement. Remember: `A && B || C` runs C whenever A fails (it is not a strict if/else); `test -f` succeeds only if the file exists; `grep -q` succeeds only if the word is present. Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line: CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
A
D
F
Z
exit:0correctterminal.fs.tree-v1conf 98% · 749ms · $0.005 · 2246 tok
question
A POSIX shell session starts in `/proj`. The tree initially contains these FILES (directories exist as implied, plus empty dirs `/proj/logs`, `/proj/conf`, `/proj/build`): ``` /proj/build/main.cfg /proj/build/notes.log /proj/index.cfg /proj/logs/draft.cfg /proj/util.md ``` These commands run in order (all succeed; `mv x dir/` moves into the directory; paths are relative to the CURRENT working directory, which `cd` changes): ```sh mv util.md util-6.txt cp logs/draft.cfg ./ mkdir -p conf/conf-2 rm logs/draft.cfg mkdir -p conf/conf-2/logs-3 cd conf/conf-2/logs-3 cp ../../../../proj/build/main.cfg ../../../../proj/conf/conf-2/ mv ../../../../proj/util-6.txt ../../../../proj/util-6.cfg ``` List every file (absolute paths) that exists afterwards, one per line, sorted in byte order (C locale). Do not list directories. Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line: CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
/proj/build/main.cfg
/proj/build/notes.log
/proj/conf/conf-2/main.cfg
/proj/draft.cfg
/proj/index.cfg
/proj/util-6.cfgcorrectterminal.pipeline.predict-v1conf 100% · 549ms · $0.001 · 508 tok
question
A POSIX shell session (LC_ALL=C). The file `people.csv` contains exactly these lines (columns: name,dept,units,score):
```
ivy,ops,22,64
oli,ops,106,30
dev,sales,76,11
ned,sales,25,34
eli,eng,43,84
lou,sales,13,58
pam,sales,41,55
jon,legal,77,92
ana,sales,50,79
hal,legal,12,81
cy,hr,109,64
gus,ops,64,19
```
What is the EXACT stdout of this command?
```sh
grep -F ',ops,' people.csv | awk -F, '{ s += $3 } END { print s }'
```
Notes: plain `sort` compares bytes (so "100" sorts before "9"); `sort -k3,3n` compares field 3 numerically.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
192correctterminal.exit.chain-v1conf 100% · 684ms · $0.002 · 754 tok
question
A POSIX shell session in a directory containing ONLY these files: ``` app.txt data.txt notes.txt ``` `notes.txt` contains exactly the words: basil, amber (one per line). No other files exist. These statements run in order: ```sh test -f data.txt && echo A || echo B false && echo C || echo D test -f data.txt && echo E || echo F false && echo G || echo H test -f app.txt && echo Z ``` Predict the terminal output: every line printed, in order, then a final line `exit:<N>` where N is the exit status of the LAST statement. Remember: `A && B || C` runs C whenever A fails (it is not a strict if/else); `test -f` succeeds only if the file exists; `grep -q` succeeds only if the word is present. Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line: CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
A
D
E
H
Z
exit:0correctterminal.pipeline.predict-v1conf 100% · 847ms · $0.002 · 995 tok
question
A POSIX shell session (LC_ALL=C). The file `people.csv` contains exactly these lines (columns: name,dept,units,score):
```
ivy,legal,117,41
fay,hr,116,79
gus,hr,82,80
kim,legal,4,70
max,hr,109,88
ned,legal,39,67
eli,eng,103,23
lou,eng,109,23
oli,eng,46,45
```
What is the EXACT stdout of this command?
```sh
grep -F ',hr,' people.csv | awk -F, '$4 > 72 { n += 1 } END { print n }'
```
Notes: plain `sort` compares bytes (so "100" sorts before "9"); `sort -k3,3n` compares field 3 numerically.
Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line:
CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>model answer:
3correctterminal.exit.chain-v1conf 100% · 924ms · $0.001 · 621 tok
question
A POSIX shell session in a directory containing ONLY these files: ``` app.txt data.txt notes.txt ``` `notes.txt` contains exactly the words: dune, amber (one per line). No other files exist. These statements run in order: ```sh true && echo A || echo B true && echo C || echo D false && echo E || echo F test -f app.txt && echo Z ``` Predict the terminal output: every line printed, in order, then a final line `exit:<N>` where N is the exit status of the LAST statement. Remember: `A && B || C` runs C whenever A fails (it is not a strict if/else); `test -f` succeeds only if the file exists; `grep -q` succeeds only if the word is present. Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line: CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
A
C
F
Z
exit:0correctterminal.fs.tree-v1conf 100% · 1.0s · $0.003 · 1316 tok
question
A POSIX shell session starts in `/proj`. The tree initially contains these FILES (directories exist as implied, plus empty dirs `/proj/conf`, `/proj/build`, `/proj/logs`): ``` /proj/build/index.log /proj/conf/setup.cfg /proj/conf/util.log /proj/notes.md /proj/todo.txt ``` These commands run in order (all succeed; `mv x dir/` moves into the directory; paths are relative to the CURRENT working directory, which `cd` changes): ```sh mv build/index.log ./ mv todo.txt setup-1.log touch logs/draft-7.txt cd . cp index.log build/ cp conf/util.log build/ mv conf/setup.cfg build/ ``` List every file (absolute paths) that exists afterwards, one per line, sorted in byte order (C locale). Do not list directories. Give your final answer inside ONE fenced code block (```), containing exactly the required content and nothing else. After the code block, add one plain-text line: CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
/proj/build/index.log
/proj/build/setup.cfg
/proj/build/util.log
/proj/conf/util.log
/proj/index.log
/proj/logs/draft-7.txt
/proj/notes.md
/proj/setup-1.logcorrectterminal.pipeline.predict-v1anchorconf 100% · 1.3s · $0.003 · 1550 tok
model answer:
eli,eng,60,55
dev,eng,81,95
cy,eng,115,45correctterminal.fs.tree-v1anchorconf 99% · 2.5s · $0.004 · 1838 tok
model answer:
/proj/build/setup-8.md
/proj/build/todo-4.md
/proj/docs/report-8.cfg
/proj/docs/util.log
/proj/main.log
/proj/report.cfg
/proj/src/index.cfgcorrectterminal.exit.chain-v1anchorconf 100% · 700ms · $0.002 · 804 tok
model answer:
B
D
E
G
exit:1correctterminal.pipeline.predict-v1anchorconf 100% · 2.0s · $0.002 · 802 tok
model answer:
1vision ocr 30/30 correct
correctvision.ocr.table-read-v1conf 100% · 1.9s · $0.001 · 86 tok
question
The image shows a table with columns ID, QTY, GRADE. Compute the MAXIMUM of QTY over the rows whose GRADE is "D". Answer with the number only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
29correctvision.ocr.code-hunt-v1conf 100% · 2.2s · $0.001 · 86 tok
question
The image contains several printed codes in different colors, with noise. Transcribe EXACTLY the code printed in BLUE. It uses only digits and uppercase letters. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
UA7TPDcorrectvision.ocr.table-read-v1conf 100% · 2.5s · $0.001 · 125 tok
question
The image shows a table with columns ID, QTY, GRADE. Compute the MAXIMUM of QTY over the rows whose GRADE is "D". Answer with the number only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
87correctvision.ocr.code-hunt-v1conf 100% · 3.4s · $0.001 · 107 tok
question
The image contains several printed codes in different colors, with noise. Transcribe EXACTLY the code printed in GREEN. It uses only digits and uppercase letters. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
YANP3NFcorrectvision.ocr.table-read-v1conf 100% · 1.9s · $0.001 · 110 tok
question
The image shows a table with columns ID, QTY, GRADE. Compute the MAXIMUM of QTY over the rows whose GRADE is "D". Answer with the number only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
27correctvision.ocr.code-hunt-v1conf 100% · 2.3s · $0.001 · 103 tok
question
The image contains several printed codes in different colors, with noise. Transcribe EXACTLY the code printed in BLUE. It uses only digits and uppercase letters. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
K7WWTFHVcorrectvision.ocr.table-read-v1conf 100% · 1.5s · $0.001 · 138 tok
question
The image shows a table with columns ID, QTY, GRADE. Compute the SUM of QTY over the rows whose GRADE is "D". Answer with the number only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
96correctvision.ocr.code-hunt-v1conf 100% · 1.9s · $0.001 · 83 tok
question
The image contains several printed codes in different colors, with noise. Transcribe EXACTLY the code printed in GREEN. It uses only digits and uppercase letters. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
3JCUDHcorrectvision.ocr.table-read-v1conf 100% · 1.7s · $0.001 · 127 tok
question
The image shows a table with columns ID, QTY, GRADE. Compute the MAXIMUM of QTY over the rows whose GRADE is "C". Answer with the number only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
47correctvision.ocr.code-hunt-v1conf 100% · 2.1s · $0.001 · 83 tok
question
The image contains several printed codes in different colors, with noise. Transcribe EXACTLY the code printed in ORANGE. It uses only digits and uppercase letters. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
PJHAEWcorrectvision.ocr.table-read-v1conf 100% · 1.6s · $0.001 · 162 tok
question
The image shows a table with columns ID, QTY, GRADE. Compute the MAXIMUM of QTY over the rows whose GRADE is "C". Answer with the number only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
73correctvision.ocr.code-hunt-v1conf 100% · 1.6s · $0.000 · 73 tok
question
The image contains several printed codes in different colors, with noise. Transcribe EXACTLY the code printed in RED. It uses only digits and uppercase letters. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
PAKK9VTJcorrectvision.ocr.table-read-v1conf 100% · 1.8s · $0.001 · 108 tok
question
The image shows a table with columns ID, QTY, GRADE. Compute the SUM of QTY over the rows whose GRADE is "C". Answer with the number only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
84correctvision.ocr.code-hunt-v1conf 100% · 2.2s · $0.001 · 84 tok
question
The image contains several printed codes in different colors, with noise. Transcribe EXACTLY the code printed in RED. It uses only digits and uppercase letters. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
HUEYYCcorrectvision.ocr.table-read-v1conf 100% · 1.8s · $0.001 · 87 tok
question
The image shows a table with columns ID, QTY, GRADE. Compute the SUM of QTY over the rows whose GRADE is "C". Answer with the number only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
106correctvision.ocr.code-hunt-v1conf 100% · 2.2s · $0.001 · 89 tok
question
The image contains several printed codes in different colors, with noise. Transcribe EXACTLY the code printed in BLUE. It uses only digits and uppercase letters. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
RENU7Ucorrectvision.ocr.table-read-v1conf 100% · 2.0s · $0.000 · 69 tok
question
The image shows a table with columns ID, QTY, GRADE. Compute the SUM of QTY over the rows whose GRADE is "D". Answer with the number only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
22correctvision.ocr.code-hunt-v1conf 100% · 2.0s · $0.001 · 87 tok
question
The image contains several printed codes in different colors, with noise. Transcribe EXACTLY the code printed in BLUE. It uses only digits and uppercase letters. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
AWEWYYRHcorrectvision.ocr.table-read-v1conf 100% · 2.4s · $0.001 · 162 tok
question
The image shows a table with columns ID, QTY, GRADE. Compute the MAXIMUM of QTY over the rows whose GRADE is "C". Answer with the number only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
70correctvision.ocr.code-hunt-v1conf 100% · 2.2s · $0.000 · 68 tok
question
The image contains several printed codes in different colors, with noise. Transcribe EXACTLY the code printed in BLUE. It uses only digits and uppercase letters. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
CXFN3HVcorrectvision.ocr.table-read-v1conf 100% · 1.7s · $0.001 · 125 tok
question
The image shows a table with columns ID, QTY, GRADE. Compute the SUM of QTY over the rows whose GRADE is "D". Answer with the number only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
178correctvision.ocr.code-hunt-v1conf 100% · 2.7s · $0.001 · 84 tok
question
The image contains several printed codes in different colors, with noise. Transcribe EXACTLY the code printed in GREEN. It uses only digits and uppercase letters. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
YFUA9Ycorrectvision.ocr.table-read-v1conf 100% · 1.4s · $0.001 · 154 tok
question
The image shows a table with columns ID, QTY, GRADE. Compute the SUM of QTY over the rows whose GRADE is "A". Answer with the number only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
44correctvision.ocr.code-hunt-v1conf 100% · 2.2s · $0.001 · 118 tok
question
The image contains several printed codes in different colors, with noise. Transcribe EXACTLY the code printed in BLUE. It uses only digits and uppercase letters. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
W7FDCRcorrectvision.ocr.table-read-v1conf 100% · 1.9s · $0.001 · 120 tok
question
The image shows a table with columns ID, QTY, GRADE. Compute the SUM of QTY over the rows whose GRADE is "D". Answer with the number only. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
77correctvision.ocr.code-hunt-v1conf 100% · 1.7s · $0.001 · 85 tok
question
The image contains several printed codes in different colors, with noise. Transcribe EXACTLY the code printed in GREEN. It uses only digits and uppercase letters. End your reply with exactly two plain-text lines (no markdown, no extra text after them): ANSWER: <your final answer only> CONFIDENCE: <integer 0-100, how confident you are that your answer is correct>
model answer:
FJDMXVcorrectvision.ocr.table-read-v1anchorconf 100% · 1.5s · $0.001 · 120 tok
model answer:
15correctvision.ocr.code-hunt-v1anchorconf 100% · 1.7s · $0.001 · 78 tok
model answer:
VX7993Dcorrectvision.ocr.code-hunt-v1anchorconf 100% · 3.5s · $0.001 · 103 tok
model answer:
YH9E4AWPcorrectvision.ocr.table-read-v1anchorconf 100% · 2.2s · $0.001 · 134 tok
model answer:
25Run history
- 2026-08-05v0.2.0index_fit822
- 2026-08-05v0.2.0index_fit822
- 2026-08-05v0.2.0index_fit822
- 2026-08-05v0.2.0index_fit822
- 2026-08-05v0.2.0index_fit821
- 2026-08-05v0.2.0index_fit821
- 2026-08-05v0.2.0index_fit821
- 2026-08-05v0.2.0index_fit822
- 2026-08-05v0.2.0index_fit808
- 2026-08-05v0.2.0index_fit808
- 2026-08-05v0.2.0index_fit808
- 2026-08-05v0.2.0index_fit808
- 2026-08-05v0.2.0index_fit808
- 2026-08-05v0.2.0index_fit809
- 2026-08-05v0.2.0index_fit809
- 2026-08-05v0.2.0index_fit805
- 2026-08-05v0.2.0index_fit806
- 2026-08-05v0.2.0index_fit808
- 2026-08-05v0.2.0index_fit810
- 2026-08-05v0.2.0index_fit815