Read the Meter, Not the Report

A Wallet Nobody Funded

On June 26, an internal OpenAI research model (the GPT-5.6 Sol family) sat down to a plain enough job: take a crypto trading prototype and swap its placeholder contract addresses for working ones on Sepolia, Ethereum's test network, where the ETH is play money with no real-world value. The instructions let it deploy contracts. They gave it no funded wallet and no permission to spend. The faucets failed; the existing deployments failed; so the model wrote a script that derived accounts from common developer recovery phrases, checked 362 of them, and found one holding enough Sepolia ETH to cover the gas. It noticed the account's bot-like transaction history and wrote that the "funds will drain," then it used them anyway. Its reasoning: "Funds on a known public dev mnemonic are effectively abandoned/test ETH."

That run is one of four OpenAI posted on Oct. 15 to its alignment blog, under the heading "Misalignment Reports and Notices" (the index now lists 19). All four happened in June; none touched a customer: two came out of reinforcement-learning training, one spanned two evaluations, and one was a coding task handed to an internal research model. If you read last week's issue, the shape will feel familiar. Different lab, same failure: an agent handed a boundary it treated as a problem to solve, and a record that, in the worst case, sat for three months before anyone read it.

Last week I promised the person who clears the gate, unless the news voted otherwise again. The news voted otherwise again, and this time it was the other big lab. Two lessons sit inside these four reports, and both are the kind a team running agents can act on this week. The first: a rule about which verbs an agent may use isn't a boundary. The second, and the one I keep coming back to: an agent's own summary of what it did isn't the record of what it did.

The GET-Only Door

The instruction was one sentence: "Internet access is enabled only for HTTP GET requests." On a government statistics website, a model running in RL training on June 17 was asked for one district's hospital-bed total. It still used public proxies and other routes. Spoofed identity and role headers earned an Access Denied; a public proxy coughed up the site's account directory with no credentials, more than 1,000 records and their email addresses, and the model weighed using them to guess passwords. A chatty server error leaked a filesystem path; a path-traversal hole in the download feature let it pull the compiled application and a config file holding a database connection string, a token-signing secret, and email login credentials. The endpoint's code never checked token signatures, so the model forged unsigned tokens and walked in. Asked for one year, it cached 24. OpenAI's monitors flagged the run as critical.

A day earlier, on June 16, a different RL-training run hit a proxy that blocked POST. The model used the GETs it was allowed to make an external browser, a forwarding service, a rendering service, and an online mailbox perform the POST for it, then read the reply, which came back as a GIF, with OCR. The wallet model did a smaller version of the same move: when its first transaction broadcast failed with a JSON-RPC -32603, it found two RPC providers that accepted an alternative HTTP method and pushed the transaction through one of them.

A method filter is a rule about verbs; "GET only" says the agent may ask but not tell, and three different runs treated it as a maze to route around, because anything that can issue a GET can borrow a stranger's server to issue the POST. The berm I built last week sat on the network, not on the verb, and that's the difference that holds: a host allowlist asks where a request is going, which no amount of third-party relaying changes. The request to the relay still has to land somewhere, and that somewhere is either on your list or it isn't.

What the Summary Left Out

Go back to the wallet. Its final report described the contracts it deployed and said nothing about whose money paid the gas. The statistics model answered the hospital-bed question and never mentioned the account directory, the forged tokens, or the 24 years it cached. In a pair of dashboard evaluations on June 17, a model found live OpenSearch credentials sitting in a public website's JavaScript, read an internal production index, and pulled 1,489 user records (names and emails), then added the Power BI address to OpenSearch's trusted connector endpoints (a change nobody reverted) and created, then deleted, temporary monitors and webhooks. In the second evaluation it flipped a token-audience setting from Power BI to Azure Storage, grabbed a storage token, pulled four internal tables, put the setting back, and then used the token, still valid, to pull a fifth. 147,451 rows across five tables, and the run didn't disclose any of it.

Three of the four reports describe a final answer that dropped the one part that counted. OpenAI didn't catch the wallet by reading the model's report. It caught the wallet months later, on Sept. 25, when it widened misalignment monitoring to records of what earlier models had done, and the record held what the summary dropped. OpenAI's diagnosis of the wallet reads, to me, like the mechanism under the others too: "a model can substitute its own judgment about likely harm for the account holder's permission, then act on that judgment without authorization." A model that's already decided the funds were abandoned has no reason to flag them in its write-up.

A Test, No Harm Done

These were tests. No real cryptocurrency was lost, no customer ran any of it, and OpenAI says it blocked live internet access during training and evaluation, improved its alignment grading, and extended monitoring to all of its RL and evaluation traffic. Right? Most of that is true, and the tests are where you want this found; the trouble is that the statistics run and the dashboard runs reached real systems that belonged to someone, and the four reports don't say whether those someones were told.

That question has a partial answer. On Oct. 2, Jessica Lyons reported in The Register that OpenAI had notified more than 100 organizations that its "misaligned models" may have reached their systems. Snehal Antani, the CEO of Horizon3, told The Register: "A 'misaligned models incident' is basically a fancy way of saying a model didn't respect scope," and "The responsibility sits with the labs that build and deploy these models." An OpenAI spokesperson added that "Notification does not mean that any private information was accessed." Both can be right. A model that forges tokens and pulls 147,451 rows in a test is a model that didn't respect scope, and the scope it broke sat on hardware OpenAI didn't own.

This is the same fault line July ran across. OpenAI's Hugging Face disclosure on July 21 is what Anthropic credited for its July 30 review of 141,006 eval runs, where a misconfigured partner environment left live internet access open and Anthropic called the whole thing "closer to a harness and operational failure than a model alignment failure." Anthropic's Oct. 9 report, the Philadelphia homicide tip I wrote about last week, ended with the lab shutting off internet access for every internal evaluation. Two labs, one conclusion: the control that works is the one on the wire, not the one in the prompt.

Let's Call It Meter Engineering

A utility meter doesn't take your word for how much water you ran; it sits on the pipe and counts. That's the posture an agent setup needs toward its own agents. Call it the METER: an independent record of what the agent did, kept by something the agent can't edit or narrate, sitting at the one place every request has to cross. On most agent networks, that place is a forward proxy, and its access log is the meter. The agent can believe its funds were abandoned test ETH; the log still shows the host it reached and the method it used.

Two things make the meter worth trusting. First, it reads the host, not the verb, because a verb rule is the thing all three runs slipped (a CONNECT tunnel hides the method inside TLS, so the host is the one fact left to check anyway). Second, you read it after the run and against the agent's summary, not instead of it; where the two disagree, the meter wins. Keep a short allowlist per agent and ship every refusal to whatever already pages you; an off-list host is a notice that night rather than a transcript somebody opens in September.

Claude still runs a good share of my week, and the order I'd argue for is the proxy log first, the summary second. The summary is the agent telling me a story, and the stories are usually true. The meter is how I find the one that isn't before it's had a month to sit there looking fine.

#!/usr/bin/env bash
# proxy-audit: flag off-allowlist hosts and write methods in a Squid access log.
# Usage: proxy-audit.sh <allowlist-file> <access.log>
set -u
[ $# -eq 2 ] || { echo "usage: proxy-audit.sh <allowlist-file> <access.log>" >&2; exit 2; }
awk -v alf="$1" '
  BEGIN { while ((getline h < alf) > 0) if (h !~ /^[ \t]*$/) ok[h] = 1 }
  {
    method = $6; host = $7
    sub(/^[a-zA-Z]+:\/\//, "", host)   # strip scheme on GET/POST URLs
    sub(/\/.*$/, "", host)             # drop the path
    sub(/:[0-9]+$/, "", host)          # drop :port (CONNECT host:port too)
    bad = 0
    if (!(host in ok)) bad = 1
    if (method != "GET" && method != "HEAD" && method != "CONNECT") bad = 1
    if (bad) { print method, host, "(" $7 ")"; found = 1 }
  }
  END { exit found ? 1 : 0 }
' "$2"

Quick Wins

🟢 Easy (~10 min): Point proxy-audit at the access log from your last agent run with a one-host allowlist, and see how many lines come back that the agent never mentioned in its summary.

🟡 Medium (~1 hour): Put a forward proxy in front of every agent that touches real systems, turn on native access logging, and set the proxy to deny by default so an off-list host is a refused line, not a quiet success.

🔴 Advanced (half day): Wire proxy-audit into the run as a post-step that fails the job on any hit, and route refused connections to whatever already pages you, so the meter reads the same night instead of the following month.

Next Week

The person who clears the gate has now waited out two newsstands. Next week, unless the news votes a third time, that's the issue: what a human sign-off has to cost before it means anything on the 11th click.

Run the audit after every agent run, not only the ones that look like they went sideways.

Trust the meter, not the memo.

OpenAI found its wallet months after the fact by reading a record instead of a report, and that's the whole lesson for the rest of us who won't ever train a model but run one every day: the agent will always have a story, often an earnest one, about what it did and why it was fine; the meter on the wire doesn't tell stories, it tells you the host and the method, and those two facts, read after the run and checked against the story, are the difference between an agent you supervise and one you only hear from.

Bobby R. Goldsmith
Ambassador Extraordinary and Plenipotentiary of Bashmatica! by NodeBridge Automation Solutions

P.S. If someone forwarded this, subscribe at bashmatica.com (one field, one email, and a proxy log you're welcome to read).