Own the memory, not just the model: the harness layer that quietly traps you

Overview

Why now: this March, Google added a button to Gemini that imports your memory from ChatGPT or Claude; in June, OpenAI rebuilt ChatGPT’s memory so it compiles what it knows about you automatically, in the background. The two features point in opposite directions, but they share one assumption worth noticing — the memory your assistant builds about you is becoming the thing that keeps you.

The valuable AI skill is no longer prompting; it’s harness-building — assembling the files, tools, checks and memory around the model. By the end you’ll have a one-sentence test for whether you own your memory or merely rent it, and a move you can make today.

The content

A model gives you intelligence; a harness gives you the work around it. As raw intelligence gets cheap — its cost is falling on the order of 10x a year — the value moves to the harness, which is where the real skill now lives. Most of a harness is replaceable: you can change tools, rewrite an instruction, switch models. Memory is the exception, because it accretes. Every week you use a tool it learns a little more about how you work, and walking away gets a little more expensive. That isn’t a flaw in any one product; it’s the shape of the thing.

So here’s the test — the exit test: if you switched tools tomorrow, could you take what the assistant has learned about you, in a form you could actually use somewhere else? “Export my data” doesn’t count — that gives you a transcript of old chats, not the distilled memory. As of mid-2026 the honest answer on most tools is no. At best you can prompt the model to write its memories out as text and keep that yourself; at worst, one major assistant now concedes that the memory summary it shows you isn’t even complete. The point isn’t to distrust a particular vendor — it’s to notice that the part of the system that compounds is the part you can least easily take with you.

The move that follows is small and entirely in your hands: don’t let the vendor’s store be the only copy of what matters. Keep your durable context — how you work, your standing instructions, the facts you’d hate to re-teach from scratch — in a file you own, and feed it in. Then the tool’s memory is a convenience, not a hostage.

Try it

On the tool you use most for real work, run this today:

Write out, verbatim, everything you currently remember about me and how I work —
my role, my preferences, my recurring projects, my standing instructions.
List it as plain text I can copy. Don't summarise it and don't reassure me it's
complete — just give me the raw contents of your memory of me.

Look at two things. First, how thin — or how surprisingly complete — it is. That is the asset you’d lose the day you switch. Second, paste the result into a file you control and keep adding to it; that file, not the vendor’s memory, becomes your source of truth, and you can carry it to any model.

Where this breaks: the export is just text, so you’ll still re-tune when you move — owning your context lowers the switching cost, it doesn’t zero it. And this isn’t an argument against using your tool’s memory; it’s an argument against ever being unable to leave it.

Additional reading

• Harness Engineering — Birgitta Böckeler, martinfowler.com, April 2026 — surfaces the shorthand “Agent = Model + Harness” (which she credits to LangChain), and why the skill has moved to the harness.
• AI Agents and Memory — New America / OTI, Nov 2025 — the “context flywheel”: the more a system remembers, the harder it is to leave.
• Memory FAQ — OpenAI Help Center and Import your memory into Gemini — Google, March 2026 — the current state of assistant memory, and why the path out lags the path in.
• AI Agent Memory Interoperability Community Group — W3C, June 2026 — the early effort to make memory portable across vendors and models.

Editor’s note

I keep an extensive, deliberately-built record of my own context: how I work, what matters to me, what I expect from these tools. I have put a great deal of thought and effort into it, and it is updated, synthesised, and maintained with every session I run. I see the value of it accrue. That is why this piece is not abstract for me. I know what an accumulated context layer is worth, which is precisely why I would not let mine live somewhere I could not take it back. Yours does not need to be large to be worth keeping; it needs to be current, and it needs to be yours. I would encourage everyone to start building one today.