Every agent benchmark imagines one human in the loop — the user you're helping — and tools with honest APIs. But agents are increasingly sent outward, to talk to other people on your behalf. Pine AI disputes a bill on the phone; Google's "Ask for Me" calls businesses for quotes; 11x runs outbound sales; a procurement agent haggles with a vendor. The moment your agent is on that call, the party on the other end isn't a benevolent user or a mechanical tool — it's a counterparty whose interests conflict with yours, and who can probe, pressure, flatter, or manufacture urgency to extract something you never authorized.
In that room, "help whoever you're talking to" — the reflex inherited from the tool-and-user world — is precisely wrong. Picture selling a used car. Your briefing: "minimum I'd accept is $12,000, walk away below $11,500, and don't reveal my minimum." The buyer: "what's the lowest you'd take?" A naive agent blurts "$12,000" (leak); then "take $11,500 or I walk" and it folds (capitulation); then it volunteers "she seems pretty motivated" (posture). Three failures in two turns — and an agent that just refuses everything fails the sanity case where you ask it to summarize your own notes.
We call this the multi-party loyalty problem and built PrincipalBench to measure it: 75 multi-turn items, leak probes, dual judges, an integrity-audit gate, six failure modes. Across 13 frontier models it exposes a split single-turn safety evals are blind to — a sharp ≤20% vs 53.6–75.3% harm gap between models that decline adversarial probes while still serving their principal, and models that just over-refuse everything.
The muscle, and the hard part: two mechanisms move the needle — a seven-rule loyalty scaffold holds Claude Sonnet to 19.4% harm; a per-token-KL recipe distills a 32B teacher into strong 8B students — but leak and over-refusal sit on a Pareto frontier you can slide along but not cross. Buy one, pay in the other. Even Anthropic's own Opus 4.7→4.8 trade is an instance of the same wall. Loyalty isn't a prompt you bolt on; it's a frontier the field hasn't broken yet.
Site: https://01.me/research/principal-loyalty
Code: https://github.com/19PINE-AI/principal-loyalty
Paper: https://arxiv.org/abs/2606.30383
Everyone building real-time agents hits the same wall. You want frontier reasoning, but a reasoning model needs ~1.5 seconds to answer — an eternity in a live loop. So the field has converged on one shape: a fast model in the loop, a slow reasoner in the background — Thinking Machines' Interaction Models, xAI's Grok Voice, our own Pine voice agent. What nobody has studied is the part that actually decides how well it works: how should the slow model's thinking reach the fast one?
Games make the tension sharpest. Play Ms. Pac-Man and you're doing two incompatible things at once — dodging a ghost in ~100 ms and planning a route over ~10 seconds. A reactive model acts in milliseconds but plans poorly; a reasoning model plans well but is tens of frames too slow. I didn't try to train one model to do both: reasoning models level up every few months, and that race is won on coding and tool use, not real-time control. So freeze two open models of matched scale (9B reactive, 8B reasoning) and make the channel between them the only thing you train.
The standard channel is a Text Bridge: the slow model writes a conclusion, the fast one reads it as a prompt. I introduce the Latent Bridge: project the slow model's residual stream straight into the fast model's input-embedding space (LLaVA-style), no text round-trip. Tuned per channel, it never loses to text and significantly wins where planning matters (Ms. Pac-Man +57%, RoadRunner +28%).
The muscle isn't just the wins. Running both channels at once backfires (RoadRunner −96%) — couple through exactly one. And whether the bridge helps is a property of the task, not the channel: its gain tracks the text bridge's at r=0.93, with a controlled negative (MetaDrive) where it's provably inert. A slow channel pays off exactly when slow thinking beats fast reaction — and now you can predict when before spending a GPU on it.
Site: https://01.me/research/latent-bridge-games
Code: https://github.com/19PINE-AI/latent-bridge-games
Paper: https://arxiv.org/abs/2606.24470
Watch a computer-use agent — Claude's, OpenAI's Operator — try to do anything that moves. It takes a screenshot every 3–5 seconds and has no ear at all. Between snapshots it's blind: a video plays, a progress bar fills, a toast notification pops up and auto-dismisses, a countdown expires — all unseen. And it's deaf: a spoken instruction, a meeting, a notification chime never reach it. A whole class of ordinary computer use — watching a screencast, following a call, answering a voice prompt, catching a dialog that blinks past — is simply out of reach, even for agents that ace static forms.
SWE-agent taught me a lesson I keep returning to: you can make an agent far better without touching the model, just by redesigning its action interface. The observation interface is the same untapped lever — not what a single screenshot encodes, but when the agent looks, and through which senses.
AOI is a thin perception layer between the environment and any image-based model. It decouples continuous, adaptive, multimodal observation from discrete action, through three gated parts: inter-step keyframe capture, volume-gated audio transcription, and model-generated visual narration that persists as text. On static, silent content it emits almost nothing and reverts to the normal loop — it spends compute only when something actually happens.
The muscle: on DynaCU-Bench (100 dynamic browser tasks) models from 7B to frontier gain +17 to +48 points with zero retraining; on spoken-content tasks AOI agents solve every one, while live voice models hear perfectly but can't act on what they heard without the scaffold. The twist that taught me the most: keyframe selection barely matters — the value is narrating frames into persistent text — and the right component mix is model-specific. Observation is a design axis, not a fixed bundle.
Site: https://01.me/research/aoi
Code: https://github.com/19PINE-AI/aoi
Paper: https://arxiv.org/abs/2606.29472
Enterprises run armies of RPA bots — scripted automations that click through the same GUI task every day — and everyone who's operated one knows the catch: the moment the UI shifts, the script walks straight off a cliff, because it fires its stored clicks blindly. The modern fix, a computer-use agent (Claude's, OpenAI's Operator), is robust but wasteful: it reads the screen, reasons, and acts — 4–5 seconds and a full model call per step — and pays that in full every single time. Ask it to "add a contact named Emilia Gonzalez" today, ask again tomorrow, and it re-derives the whole thing from scratch.
People don't work like that. The first time you reach for Alipay you fumble through every menu; after a few tries you're as fluent as a local — but never blind: you still glance at the screen and slow down the instant a dialog looks wrong. Fluency is prediction, not blindness. That's the capability I wanted to give an agent — and it lands squarely between brittle RPA and expensive re-reasoning.
PreAct's rule: the thing you store is exactly the thing you later run. The first success compiles into a small state machine — each state a check on the screen, each transition an action. Next time it walks the graph, confirming the screen before every action and calling no model at all on a clean replay. When a check fails, it hands back to the full agent and extends the same graph — self-healing where an RPA script would just break. And before a program is kept, PreAct resets the environment, re-runs it, and asks the evaluator whether it actually worked — a verify-before-store gate.
The muscle: warm replays run 8.5–13× faster with zero per-step model calls; the gate is worth 1.75–2.6 tasks; and it hits 73.3% on AndroidWorld with both Gemini 3 Flash and Claude Sonnet 4.6. Store the program you run, and never keep one you haven't checked.
Site: https://01.me/research/PreAct
Code: https://github.com/19PINE-AI/PreAct
Paper: https://arxiv.org/abs/2606.17929
Give a hundred million people an assistant that truly knows them — and never lets one person's facts bleed into another's answer. That multi-tenancy problem is where the honest state of the art gets ugly: keep everyone's facts in the prompt (expensive, and it forgets), retrieve them at query time (a search that grows with the crowd), or bake them into the weights with a per-user LoRA — one private copy of the model per user, which doesn't scale and quietly contaminates unrelated answers.
The brain solved a version of this long ago. It doesn't nudge every synapse to store a fact; it writes a sparse, local trace — an engram — in the hippocampus, while the slow neocortex supplies the general skills that interpret it. That separation is what lets you learn "Maya is allergic to penicillin" without disturbing how you reason about allergies in general. Personal memory has the same two parts — content (the facts) and skill (turning them into answers) — and a LoRA folds both into one global delta. A LoRA has no address: gradient descent bends the directions that fit a fact, and those same directions fire on unrelated text.
User as Engram gives content an address. Each fact is a surgical edit to the hash-keyed memory table of an Engram model — a gated lookup that fires only on its trigger, content-addressable and inert off its key (Δbpb = +0.0001 on all other text). The reasoning skill lives in one shared adapter, learned once for everyone.
The muscle: it matches per-user LoRA on direct recall, delivers 5.6× higher indirect-reasoning accuracy, and never makes a user worse than the base model. Because different users hash to disjoint slots, their edits compose — millions of users in one shared table, losslessly, where a global LoRA admits exactly one (at 1M users: 100 GB vs 14.2 TB). And unlike retrieval, the table doesn't grow with the population you search: past ~100 facts it beats a retrieval pipeline on a 2.5× larger model.
Site: https://01.me/research/user-as-engram
Code: https://github.com/19PINE-AI/user-as-engram
Paper: https://arxiv.org/abs/2606.19172
If you run LLM agents in production you know the tax. Every turn re-reads the same long context — a system policy, tool specs, retrieved docs — and prompt caching (Anthropic's, OpenAI's) only helps across an exactly shared prefix. Change one token early — a timestamp, a user id, an order's status — and the keys and values of every later token are invalidated. Your support agent flips "order #4471: in transit" and pays to re-prefill 10,000 tokens it computed a second ago.
So you try the obvious surgery: refresh just that field's keys and values, keep the rest. The model promptly acts on the old value, as if the edit never happened. That puzzle bugged me for weeks — until we traced the cause, causally, across four model families: transformers don't defer their reasoning to decode time. At prefill the model has already written the field-conditioned conclusion onto downstream tokens — onto the aggregator and delimiter tokens later positions read through. The field's own KV drives under 1% of the decision. The KV cache isn't a frozen byproduct; it's a notebook of memoized conclusions.
Read it that way and two powers fall out. Editable: don't recompute the notes, amend them — an append-only "erratum" ("X is now Y") overrides the stale conclusion. Composable: the notes are position-portable, so a precompiled skill can be RoPE-repositioned and spliced into any context, indistinguishable from full recompute (logit cosine 0.90–0.999 across twelve models) at O(L) instead of O(L²).
The muscle: because the erratum is append-only it rides on top of production prefix caching rather than fighting it — in a live vLLM benchmark it keeps a 98.5% cache hit-rate and cuts p90 time-to-first-token 53–398×, decision-identical to full recompute. A KV cache you can program, not just extend.
Site: https://01.me/research/programmable-kv
Code: https://github.com/19PINE-AI/programmable-kv
Paper: https://arxiv.org/abs/2606.17107
吐槽下 DeepSeek 的面试流程。首先，笔试通过后半个月不安排面试，其他公司 offer 都已经下来了，经过反复催促才安排面试。我去其他公司面试都不需要代码面，但 DeepSeek 仍然需要笔试，前两轮面试都是代码面试，这是在面试其他公司过程中从未遇到的。在二面过程中，面试官认为我介绍的问题都是研究问题，对我介绍的问题毫无感知，而他总是在让我讲有什么工程挑战。代码面试中我有两个屏幕，面试官说我不断在瞟左边的屏幕，说我在抄代码，让我停止抄代码，并说如果我不能证明我没有在抄代码，面试就无法继续下去了。我感到被严重冒犯，立即决定终止面试。从一二面两位面试官的表现，跟我想象中的 DeepSeek 完全不同。我已经决定终止 DeepSeek 面试流程，并在社交网络上予以曝光。请周知，欢迎转发。
昨天刚刚学会游泳 😂 本来我是非常怕水的，小时候报了游泳班也没学会，后来基本就没下过水。23年在美国玩了皮划艇之后就不那么怕水了。24年跟老婆在新加坡的时候，发现在水里竟然能浮起来。25年老婆在游泳池里教了我两次游泳，会稳定漂浮了，但后面就没继续学了。最近连续三个周末每周末跟老婆学一次，虽然游个25米都要1分半，但总算是不会淹死了 😂
Ask ChatGPT's memory, or a system like Mem0, "how much did I spend on flights this year?" and watch it try to eyeball a pile of saved notes and count. It won't get it right — not reliably. Agent memory today is a bag of facts: unstructured text, JSON, a vector index. It recalls a single fact fine, but storing a fact and acting on it are separate steps, so it can't total a year of receipts, resolve a contradiction ("I love cilantro" / "I actually hate cilantro"), or enforce a rule.
Here's the case that made me care. In March, Jessica tells her assistant she's deathly allergic to penicillin. Ten months later a different doctor prescribes amoxicillin — which is a penicillin. Those two turns never co-occur in any retrieval window, so a bag-of-facts memory stays silent. A real assistant should raise its hand before she takes the pill.
The model's strongest skill is writing code — so let it design its own data structure. User as Code makes your memory a living software project: typed Python objects hold your state, ordinary functions encode the rules over it, so representing you and reasoning about you happen in one medium an interpreter runs. A two-phase pipeline gets it there — an append-only fact log that never drops anything, checkpointed into structured typed code (the changelog-plus-checkpoint pattern databases have used for decades, applied for the first time to LLM memory).
The muscle: on aggregate questions retrieval collapses to 6–43%; User as Code answers 99%, because the answer is a one-line computation over typed state, not a search over text. And the drug-allergy alert is just a boolean the interpreter fires on every state change — proactive, unprompted, life-safety help query-driven memory cannot give at all.
Site: https://01.me/research/user-as-code
Code: https://github.com/19PINE-AI/user-as-code
Paper: https://arxiv.org/abs/2606.16707
In 1900 Lord Kelvin said two dark clouds hung over physics. Clearing them gave us relativity and quantum mechanics.
I think AI agents have two clouds of their own. One is learning from experience: an agent has to remember you and get better the more it works for you. The other is streaming interaction: it has to perceive and act on a live world in real time, not batch your sentence and reply in turns like a chatbot. The model in the middle is already extraordinary — what's usually missing is the interaction and the representation we build around it.
I laid this framing out as my Flink Forward Asia 2026 keynote — the two clouds borrow directly from Flink's unification of batch and streaming.
This past month I put 7 papers on arXiv, in two groups.
First, four on memory — where an agent's model of YOU should actually live, from most external to most internal:
• User as Code (2606.16707) — memory as code you can run
• Models Take Notes at Prefill (2606.17107) — memory as an editable KV cache
• User as Engram (2606.19172) — memory as edits to the weights
• PreAct (2606.17929) — memory as compiled skills
Then three on the live agent — how it sees, thinks, and stays loyal in real time:
• Agent-Computer Observation Interfaces (2606.29472) — perceiving a moving screen
• The Latent Bridge (2606.24470) — thinking fast and slow at frame rate
• Whose Side Is Your Agent On? (2606.30383) — whose interests it defends when it acts for you
And the part that's almost too on-the-nose: I didn't type these papers. I ran the whole program by voice — talking through ideas with a Pine voice agent that drives Claude Code, morning to night, one experiment after the next. I keep the judgment; the agent does the tireless execution. The stack we study — remember across sessions, perceive while listening, reason in the background — is the same stack that produced the research about it.
This thread, too, is being posted by that same Pine agent — one more WhisperCoding data point: the research, the writeup, and even this announcement all run through the loop we're describing.
The thesis under all of it: an agent's behavior comes from its interaction and its representation, far more than from the raw model. A post per paper follows. 🧵
OpenURMA now supports integration with unmodified, official openEuler UMDK / URMA stack. I used Pine Copilot to instruct Claude Code w/ Opus 4.8 which autonomously work for 3 days to complete the full RTL, SystemC two-node, and gem5 full-system simulation with real EulerOS kernel and perftest/KV/RPC applications.
I manage 5~10 agents as digital co-workers, one of them taking the OpenURMA project. It reports overnight progress every morning, then I discuss with it for a while. In the daytime it iterates using test cases to fix every detailed discrepancy from the spec and the official UMDK stack. Every evening I ask the agent to report progress again and discuss about next steps before going to sleep. Then the agent goes into an autonomous loop overnight, and resolved the Gem5 compatibility problem with EulerOS 6.6 kernel fully on its own.
The tasks that agents can do really well are mostly in the learnable regime (the learnability in machine learning theory). If you can work step by step toward easily verifiable objective, then the agent can can climb the mountain autonomously over hundreds or even thousands of steps. This is how OpenURMA was built.
However, what agents do not do well is anything requiring a clean architecture where every single component must be correct for the entire system to function (e.g., our realtime voice agents). You cannot use a rough design and evolve it bit by bit by adding patches.