Self-improving LLM harness

#agents #claude-code #devtools #knowledge-base #my-oss #obsidian #projects

I recently wrote about using my Obsidian vault as external memory for Claude Code. In a nutshell, I replaced Claude Code's built-in memory with my Obsidian wiki that it can reference across sessions across all my machines. I wrote an obsidian-knowledge plugin which bundles skills and hooks to get this all setup and enforced.

Since then, I've been extending it from a self-authoring memory system to an entire self-authoring agent harness. Essentially, these are a bunch of hooks that nudge the agent toward good behavior, constrain it from bad behavior, and (the new part) let the agent help author the harness itself. This is inspired by "Automated Design of Agentic Systems" (ADAS), OpenAI's Harness Engineering post, and this post about enforcing strict agent boundaries.

I implement this "harness" usingClaude Code hooks, which are scripts that can fire at a specified time: session start, before a tool call, after a tool call, or at the end of a turn. I use these hooks for different purposes:

Self-authorship hooks detects when the user is frustrated ("fuck", "omg", "are you joking", "I already told you") and fires off a subagent to do a post-mortem on the conversation up to that point. It ponders why the system failed at that moment and how to make new hooks to improve system robustness.

Nudging hooks periodically remind the agent about norms. For example, at the start of a convo it gives a quick primer about my memory system. At the end of several turns it nudges the model to reflect on any gotchas or learnings and to document them in the knowledge base. It also enables the agent to spawn subagents to run the self-authoring routine.

Constraining hooks detect bad actions and block them. For example, it can't edit the blog posts from this website (which is published from the same Obsidain vault) without me giving explicit permission. Or, if it tries to edit Claude's built-in memory files, it blocks it and makes it use my Obsidian vault.

If you want to give it a shot, take a look at the obsidian-knowledge plugin.