Introduction

A from-the-ground-up coding agent inspired by Recursive Language Models (Zhang, Kraska & Khattab, 2025). Works with any text-based model.

Vis takes a fundamentally different approach from current coding agent harnesses. Instead of accumulating messages into an ever-growing context window — then desperately compacting when it overflows — Vis treats the context as an external environment the model interacts with through code. The model writes Clojure, a sandboxed interpreter executes it, and results flow back as a compact journal. State lives in named vars and a SQLite DB, not in the token budget.

No compaction. No sliding windows. No “summarize the last 50 messages”. The model sees exactly what it needs: the previous iteration’s results, a var index of everything it has defined, and system nudges. Everything else is one function call away.

Quick Links

Rules for Contributors

Any change to source code that affects architecture, environment shape, extension spec, iteration flow, or public API MUST be accompanied by an update to these docs in the same commit. If docs and code diverge, code wins — fix the docs immediately.

Rationale

Inspired by Recursive Language Models (Zhang, Kraska & Khattab, 2025). Built from the ground up. Works with any text-based model.

The model writes Clojure. A sandboxed SCI interpreter executes it. Results flow back as a compact journal. State lives in named vars and SQLite, not in the token budget.

The Problem

Every coding agent (Claude Code, OpenCode, Pi, Hermes) runs the same loop:

sequenceDiagram
    participant U as User
    participant H as Harness
    participant LLM
    participant T as Tools

    U->>H: "Fix the auth bug"
    H->>LLM: [system, user]
    LLM->>H: assistant + tool_use: read_file
    H->>T: execute
    T->>H: result
    H->>LLM: [system, user, asst+tool_use, tool_result]
    LLM->>H: assistant + tool_use: grep
    H->>T: execute
    T->>H: result
    H->>LLM: [system, user, asst, tool_res, asst, tool_res]
    LLM->>H: assistant + tool_use: edit_file
    H->>T: execute
    T->>H: result
    H->>LLM: [sys, user, asst, res, asst, res, asst, res]
    LLM->>H: final answer

Each tool call adds 2 messages to context (assistant tool_use + tool_result). 10 tool calls = 20 messages. Context grows monotonically. ~80% of tokens end up being tool results the model may never reference again.

When context fills up: compaction. Summarize old messages, drop results. Every compaction loses signal. The model forgets, re-reads, forgets.

Other costs: one tool per round-trip (no composition), requires function_calling API support, hallucinated tool names.

How Vis Works

No tool calls. No message accumulation. No compaction.

sequenceDiagram
    participant U as User
    participant V as Vis
    participant LLM
    participant SCI as SCI Sandbox

    U->>V: "Fix the auth bug"
    V->>LLM: [system, user requirement]

    rect rgb(240, 248, 255)
        Note over V,SCI: Iteration 1
        LLM->>V: JSON {thinking, code: [read, grep, edit]}
        V->>SCI: execute all 3 blocks
        SCI->>V: 3 results
    end

    V->>LLM: [system, user req, ONE msg: journal + var_index]

    rect rgb(240, 255, 240)
        Note over V,SCI: Iteration 2
        LLM->>V: JSON {thinking, code: [run-tests]}
        V->>SCI: execute
        SCI->>V: result
    end

    V->>LLM: [system, user req, ONE msg: journal + var_index]

    rect rgb(255, 248, 240)
        Note over V,SCI: Iteration 3
        LLM->>V: JSON {final: {answer: "Fixed..."}}
    end

    V->>U: "Fixed the auth bug"

The model sees one context message per iteration:

• <journal> — previous iteration’s results (not accumulated)
• <var_index> — all named vars rendered as compact pseudo-source, e.g. (def ^{:v 3 :s :l :t :map :n 12} foo ...); :v means persisted version count and full history is available via (var-history 'sym)
• [system_nudge] — budget, repetition, extension hints
• <prior_thinking> — previous iteration’s reasoning only

Everything older is one function call away: (var-history 'x), (conversation-history).

Secure by Default

In tool-call agents, every tool has direct host access. bash runs shell commands. write_file writes anywhere. The harness adds permission prompts (“Allow write to /etc/passwd?”) or trust policies. Security is opt-in, bolted on.

Vis inverts this. The SCI sandbox is a deny-by-default environment:

• eval, load-file, spit, sh, *in*, *out* — blocked
• File system, network, shell — no access unless an extension grants it
• Java interop — only classes explicitly exposed (LocalDate, UUID, Pattern, etc.)
• Per-block timeout — infinite loops get killed

The model can only do what extensions allow. An extension that registers read-file decides the allowed paths, size caps, and tracking. An extension that registers bash decides which commands are permitted. No extension = no capability. There are no permission prompts because there is nothing to permit — the sandbox boundary is the policy.

This is why the extension system is the only way to add capabilities. It’s not a plugin architecture for convenience — it’s the security model.

Why SCI

SCI — sandboxed Clojure interpreter on the JVM. Full Clojure semantics, deny-list sandboxing, per-block timeouts, selective Java interop, persistent vars across evaluations.

What We Took From Others

Pi — extension system design (activation guards, lifecycle hooks). Pi’s extensions persist state via appendEntry() into JSONL session files — append-only, per-session, no structured queries. Vis gives extensions a shared SQLite DB with versioned snapshots, queryable across conversations.

Claude Code / OpenCode — speed matters, permissions kill flow. Vis uses the sandbox as the permission system.

Hermes — ambitious 5-layer memory architecture. But a 10K-line monolith with undocumented heuristics. Vis keeps every iteration inspectable with full provenance in SQLite.

Architecture Overview

graph TD
    subgraph Channels
        Web["Web (Jetty)"]
        TUI["TUI (Lanterna)"]
        Telegram["Telegram"]
        CLI["CLI"]
    end

    subgraph Conversation["Conversation Layer"]
        Conv["Lifecycle + locking<br/>one conversation = one environment"]
    end

    subgraph Environment["Environment"]
        Env["Runtime map for one live conversation"]
        SCI["SCI Sandbox<br/>deny-by-default execution"]
        Ext["Extensions<br/>only way to add capabilities"]
        VarIdx["Var Index<br/>persistent named state"]
    end

    subgraph Query["Query Engine"]
        QE["One user turn<br/>validate, persist, iterate"]
    end

    subgraph Iteration["Iteration Engine"]
        IE["One LLM round-trip<br/>context, ask, execute, persist"]
        Prompt["Prompt + Nudges<br/>O 1 context per iteration"]
    end

    subgraph Persistence["Persistence"]
        DB["Single SQLite DB<br/>versioned snapshots of everything"]
    end

    Web --> Conv
    TUI --> Conv
    Telegram --> Conv
    CLI --> Conv
    Conv --> Env
    Env --> QE
    QE --> IE
    IE --> Prompt
    IE --> DB
    Env --- SCI
    Env --- Ext
    Env --- VarIdx

Layer Responsibilities

Channels — external surfaces only. HTTP routes, terminal rendering, bot polling, CLI argument parsing. No business logic. Each channel calls into the conversation layer.

Conversation Layer — owns the in-process cache of live environments, per-conversation locking, and the send → query bridge. One conversation = one environment = one SCI sandbox.

Environment — the runtime map representing one live conversation. Holds the SCI sandbox, registered extensions, var-index cache, DB handle, and router. See Environment Map for every key.

Query Engine — one user turn. Validates inputs, stores the query entity, enters the iteration loop, finalizes cost/duration/tokens.

Iteration Engine — one LLM round-trip. Assembles context (journal, var-index, nudges, prior thinking), calls the LLM, executes code blocks in SCI, persists results. Context is O(1) — never grows with iteration count. The system prompt is built by loop-core/assemble-system-prompt — single source of truth shared by both loop paths and the TUI inspector.

Persistence — single SQLite DB for everything. Every (def ...), every iteration, every thinking step persisted as versioned snapshots. Full provenance for post-mortem and conversation resume.

Iteration Flow

What happens when the user sends a message, end to end.

Sequence

flowchart TD
    User(["User Message"])
    User --> Send

    subgraph Conversation["conversation/core.clj"]
        Send["send! — acquire lock, build history"]
    end

    Send --> QueryFn

    subgraph Query["query/core.clj"]
        QueryFn["query! — validate, store query, enter loop"]
    end

    QueryFn --> BuildCtx

    subgraph Loop["iteration-loop"]
        direction TB
        BuildCtx["1. Build Context"]
        AskLLM["2. Ask LLM"]
        Execute["3. Execute Code"]
        Persist["4. Persist + Decide"]

        BuildCtx --> AskLLM
        AskLLM --> Execute
        Execute --> Persist
        Persist -- "has :code, no :final" --> BuildCtx
    end

    Persist -- ":final present" --> Answer(["Answer + metadata"])
    Persist -- "errors" --> FeedBack["Feed error to LLM"]
    FeedBack --> BuildCtx

Step details:

1. Build Context — iter header, <prior_thinking>, <journal>, <var_index> (compact pseudo-source index of defs/defns), nudges (built-in + extension)
2. Ask LLM — svar structured JSON output: code blocks + optional :final
3. Execute Code — lint, SCI eval with timeout, capture stdout/stderr/result per block
4. Persist + Decide — store-iteration!, attach extension metadata, route to next step

System Prompt Assembly

loop-core/assemble-system-prompt is the single source of truth for the system message content. Both iteration loop paths and the TUI [?] inspector call it. It composes:

1. Core instructions (CORE_SYSTEM_PROMPT) — iteration steps (READ/COMPUTE/PERSIST/FINALIZE), Mustache docs, grounding rule, query primacy, perf hints, tool discipline, CLJ rules, output voice
2. Date + environment block — CWD, home, user, platform, shell
3. Extension prompts — each active extension’s :ext/prompt, prefixed with [namespace: alias → ns]

The iteration spec schema (svar’s spec->prompt) is appended separately by svar as a final user message — it is NOT part of the system message.

Error Recovery

flowchart LR
    Error["Iteration throws"]
    Error --> Classify{"Infrastructure?"}
    Classify -- Yes --> Abort["Re-throw, turn aborts"]
    Classify -- No --> Normalize["Normalize error"]
    Normalize --> FeedBack["Append as user message"]
    FeedBack --> NextIter["Next iteration"]
    NextIter --> Budget{"consecutive errors >= 5?"}
    Budget -- No --> Continue["Continue loop"]
    Budget -- Yes --> Restart{"restarts < 3?"}
    Restart -- Yes --> Reset["Strategy restart"]
    Restart -- No --> GiveUp["Budget exhausted"]

Budget Extension

The default budget is 4 iterations — deliberately tight so the LLM must plan. When more work is genuinely needed, the LLM calls (request-more-iterations n) from :code to extend on demand. There is no cap on how high the budget can grow.

This is especially important when a budget [system_nudge] fires. The intended behavior is: read the nudge, decide whether more work is actually needed, and if yes call (request-more-iterations n) immediately instead of limping into a bad finalize.

Prior Thinking

Only the most recent iteration’s :thinking is shipped in <prior_thinking>. Older reasonings are accessible on demand via (var-history '*reasoning*) from :code. This is deliberate — eager auto-context burns tokens on summaries nobody asked for.

More generally, <var_index> is only the latest namespace snapshot. When a symbol shows :v N, the full persisted version timeline is available via (var-history 'sym). This includes SYSTEM vars like *query*, *reasoning*, and *answer*.

Cross-query handover at iteration 0 ships the last 2 reasonings + final answer from the previous turn. This is a separate mechanism.

State Ownership

Lifetime Table

Environment Map

The environment is the runtime map representing one live conversation. See Environment Map for every key, its type, and what you can/cannot do with it from extension code.

Conversation Cache

conversation/core.clj maintains a process-level (defonce cache (atom {})) mapping conversation ID strings to {:env environment :lock Object}.

• ensure-env! — find-or-create in cache
• cache-env! — insert into cache
• close! — dispose environment + remove from cache
• close-all! — dispose all + reset cache (process shutdown)

The per-conversation :lock object serializes send! calls — only one turn runs at a time per conversation.

Database Schema

Single SQLite DB for everything: ~/.vis/vis.mdb/rlm.db.

Schema source of truth: resources/db/migration/V1__schema.sql.

Flyway migration location: classpath:db/migration.

Entity Tree

graph TD
    CS[conversation_soul] --> CST[conversation_state]
    CST --> QS[query_soul]
    CST --> ES[expression_soul<br/>var / call / literal]
    QS --> QST[query_state]
    QST --> IT[iteration]
    IT --> EST[expression_state<br/>code execution results]
    ES --> ED[expression_dependency<br/>directed edges]
    LOG[log] -.->|optional FKs| CS
    LOG -.-> QS
    LOG -.-> IT
    LOG -.-> ES
    SEARCH[search FTS5] -.->|triggers| QS
    SEARCH -.-> EST

Tables

1) conversation_soul

Conversation identity.

Index: idx_conv_soul_created(created_at DESC)

2) conversation_state

Forkable mutable state for a conversation soul.

Constraints: UNIQUE(conversation_soul_id, version)

3) query_soul

Immutable identity of a user ask (branch-local).

Index: idx_query_soul_state(conversation_state_id, created_at)

4) query_state

One run/retry state for query_soul.

Constraints: UNIQUE(query_soul_id, version)

5) iteration

One LLM round-trip inside a query_state.

Constraints: UNIQUE(query_state_id, position)

Iteration Metadata

The metadata column stores per-iteration context as JSON:

{"extensions": [
  {"namespace": "com.blockether.vis.ext.editing",
   "version": "0.1.0"},
  {"namespace": "com.acme.ext.git",
   "version": "2.3.0"}
]}

Records which extensions (with full source namespace and version) were active when the iteration ran, enabling post-mortem analysis and reproducibility.

6) expression_soul

Branch-local identity for var/call/literal expression nodes.

Constraints: CHECK(kind <> 'literal' OR state_mode = 'stateless')

Unique partial index: uq_expression_soul_state_name(conversation_state_id, name) WHERE name IS NOT NULL

7) expression_dependency

Directed dependency edges between expression souls.

Constraints: CHECK(downstream <> upstream), UNIQUE(downstream, upstream)

Triggers enforce same conversation_state_id across endpoints.

8) expression_state

Versioned expression output snapshots emitted per iteration.

Constraints: UNIQUE(expression_soul_id, version), CHECK((success=1 AND error IS NULL) OR (success=0 AND error IS NOT NULL))

Triggers enforce: first version = 0, stateless expressions only get version 0.

9) log

Structured logs with optional scope references.

Scoped partial indexes for each nullable FK.

10) search (FTS5)

Full-text search virtual table.

Tokenizer: porter unicode61 remove_diacritics 2

Indexed sources via triggers:

• query_soul.query
• expression_state.expr

Persistence Rules

1. All DB code lives under persistance/* — nowhere else.
2. HoneySQL only — no raw SQL outside persistance/sqlite/*.clj.
3. Callers use persistance/core.clj — never import sqlite/core.clj directly.
4. Schema changes in V1__schema.sql MUST update this page.
5. If this doc and SQL diverge, V1__schema.sql is authoritative — fix the doc.

Extension System

Namespace: com.blockether.vis.loop.runtime.conversation.environment.extension

Extensions are the only way to add symbols, classes, and documentation to the SCI sandbox. An extension is a namespace-like bundle that groups related tools, constants, prompt context, and per-iteration nudges into a single validated unit.

What an Extension Can Do

1. Bind functions into an aliased namespace - the LLM calls (alias/fn ...) from :code
2. Bind constants - data the LLM references via the alias prefix
3. Inject prompt context - LLM-facing docs in the system prompt
4. Emit per-iteration nudges - situational hints (budget, errors, etc.)
5. Expose Java classes - enable (LocalDate/now) style interop
6. Guard activation - conditionally enable/disable based on env state

Registration

Two ways to register extensions:

Global Registry (recommended)

Call register-global! at namespace load time. When any environment is created, all global extensions are automatically installed in dependency order.

(ns my.company.ext.git
  (:require [....extension :as ext]))

(ext/register-global!
  (ext/extension
    {:ext/namespace 'com.acme.ext.git
     :ext/requires  ['com.blockether.vis.ext.editing]
     :ext/doc       "Git integration"
     ...}))

Drop the jar on the classpath → namespace loads → extension self-registers → every new environment gets it.

Auto-Discovery from Classpath (recommended)

Extensions can be discovered automatically without any manual require. Place a META-INF/vis/extensions.edn file in your extension’s resources/ directory:

[com.acme.ext.git
 com.acme.ext.search]

When create-environment runs, it calls discover-extensions! which:

1. Scans the classpath for all META-INF/vis/extensions.edn files (via ClassLoader.getResources)
2. Reads each file as a vector of namespace symbols
3. requires each namespace (triggering its register-global! call)
4. Skips namespaces that are already registered
5. Logs every success at :info and every failure at :error

This means: add the extension jar/local-root to your deps.edn aliases, ensure it has a META-INF/vis/extensions.edn in its resources, and it will be loaded automatically. No imports, no requires, no wiring.

Directory layout for an extension:

extensions/my-ext/
├── deps.edn                         ;; {:paths ["src" "resources"] ...}
├── resources/
│   └── META-INF/vis/extensions.edn   ;; [com.acme.ext.my-tool]
└── src/com/acme/ext/my_tool.clj     ;; calls register-global! at load time

deps.edn alias:

:run {:extra-deps {com.acme.ext/my-tool {:local/root "extensions/my-ext"}}}

Dynamic Loading

An extension can load other extensions at runtime:

(ext/load-extension! 'my.company.ext.git)
;; => requires the ns, triggers register-global!, returns the ext

This is how meta-extensions (extension packs) work - one extension requires others dynamically.

Per-Environment (ad-hoc)

(register-extension! environment my-ext)

For extensions that shouldn’t be global.

Lifecycle

flowchart TD
    Discover["0. discover-extensions!"]
    Discover --> Build["1. ext/extension"]
    Build --> Global["2. register-global!"]
    Global --> Topo["3. Topo-sort by ext/requires"]
    Topo --> Deps{"4. Dependencies met?"}
    Deps -- yes --> Install["5. Install into environment"]
    Deps -- no --> Fail(["Throws missing-dependencies"])
    Install --> Prompt["6. Append ext/prompt to system prompt"]
    Prompt --> Activate{"7. Per-query activation-fn?"}
    Activate -- active --> Nudge["8. Per-iteration nudge-fn"]
    Activate -- inactive --> Skip(["Symbols unbound, nudge skipped"])
    Nudge --> Hooks["9. before-fn, fn, after-fn"]

Step details:

0. discover-extensions! - scan META-INF/vis/extensions.edn on classpath
1. ext/extension - build and validate extension spec
2. register-global! - add to process-level registry
3. Topo-sort - order by :ext/requires dependencies
4. Dependencies - all required extensions must be registered
5. Install - bind symbols into aliased SCI namespace, auto-require alias in sandbox
6. Prompt - append [namespace: alias → ns] header + :ext/prompt to system prompt
7. Activation - per-query activation-fn check
8. Nudge - per-iteration nudge-fn called
9. Hooks - per-call before-fn, fn, after-fn, on-error-fn

Namespace Aliases (required)

Every extension must declare :ext/ns-alias - a map with :ns (the full SCI namespace symbol) and :alias (the short alias the LLM uses). Extension symbols are bound only into this dedicated namespace, never into the sandbox namespace directly. The LLM must always use the alias prefix.

(ext/extension
  {:ext/namespace 'com.blockether.vis.ext.editing
   :ext/ns-alias  {:ns 'vis.ext.fs :alias 'fs}
   ...})

At register-extension! time:

1. A SCI namespace vis.ext.fs is created with all wrapped symbols
2. The alias fs is registered in the SCI context
3. (require '[vis.ext.fs :as fs]) is auto-evaluated in the sandbox
4. The LLM calls (fs/read-file ...), (fs/list-files ...), etc.
5. Bare (read-file ...) does not resolve - the alias is mandatory

The system prompt auto-prepends a namespace header to each extension’s prompt block:

[namespace: fs → vis.ext.fs]
Filesystem tools (use fs/ prefix):
- (fs/read-file path) ...

Extension-declared :ext/classes and :ext/imports are also injected into the SCI context, so (LocalDate/now) works if an extension exposes java.time.LocalDate.

Prompt Injection

Every active extension’s :ext/prompt is appended to the system prompt at the start of each query. This is how the LLM knows which tools are available in the sandbox.

loop-core/assemble-system-prompt is the single function that builds the complete system message. It:

1. Builds the core system prompt (CORE_SYSTEM_PROMPT + date + environment block + optional caller instructions)
2. Collects extension prompts: for each extension where (:ext/activation-fn ext) environment is truthy, (:ext/prompt ext) environment is called
3. Joins all active prompts with \n\n and appends to the core prompt

Both iteration loop paths (loop/core.clj and query/core.clj) and the TUI [?] inspector (conversation/core.clj :: effective-system-prompt) call this same function — zero duplication, zero drift.

If an extension’s activation-fn or prompt fn throws, the error is logged at :error level and that extension’s prompt is skipped — the query still runs.

Quick Example

(ns com.acme.ext.search
  (:require [c.b.vis.loop.runtime.conversation.environment.extension :as ext]))

(defn- search-fn [query] ...)

(def search-ext
  (ext/extension
    {:ext/namespace     'com.acme.ext.search
     :ext/doc           "Document search"
     :ext/group         "knowledge"
     :ext/ns-alias      {:ns 'vis.ext.search :alias 'search}
     :ext/prompt        "Document search tools (use search/ prefix):
- (search/find query) - full-text search across documents"
     :ext/symbols       [(ext/symbol 'find search-fn
                           {:doc      "Full-text search."
                            :arglists '([query])
                            :examples ["(search/find \"neural\")"]})]}))

;; Self-register at load time
(ext/register-global! search-ext)

The LLM sees in the system prompt:

[namespace: search → vis.ext.search]
Document search tools (use search/ prefix):
- (search/find query) - full-text search across documents

And calls (search/find "neural") from :code blocks. Bare (find "neural") does not resolve.

Sections

• Extension Spec - all keys, defaults, validation
• Hook Protocol - :before-fn, :after-fn, :on-error-fn
• Environment Map - every key in the environment
• Nudge System - built-in + extension nudges

Extension Spec

Auto-Discovery

Extensions are auto-discovered from META-INF/vis/extensions.edn on the classpath. See Overview — Auto-Discovery for the full convention.

extension — build and validate

(ext/extension spec) → validated extension map

symbol — function binding

(ext/symbol sym-name f opts) → validated fn symbol entry

value — constant binding

(ext/value sym-name val opts) → validated value symbol entry

wrap-extension — bind into SCI

(ext/wrap-extension ext env) → {sym → fn-or-value}

Wraps every function symbol through invoke-symbol-wrapper (before → fn → after, with on-error recovery). Value symbols are returned as {sym → value}. Each wrapped fn closes over the extension, symbol entry, and environment.

validate! — standalone validation

(ext/validate! ext) → normalized ext (or throws)

Normalizes :ext/prompt (string → fn) then checks the spec. Called internally by extension; safe to call standalone.

Note: :ext/prompt accepts string or fn?. Both extension and validate! normalize strings to (constantly s) before validation.

Full Example

(ns com.blockether.vis.ext.documents
  (:require [c.b.vis.loop.runtime.conversation.environment.extension :as ext]))

(defn- search-fn [query] ...)
(defn- search-with-opts [query opts] ...)

(def search-sym
  (ext/symbol 'search search-fn
    {:doc        "Full-text search across ingested documents."
     :arglists  '([query] [query opts])
     :examples  ["(docs/search \"neural\")"
                 "(docs/search \"attention\" {:limit 5})"]
     :before-fn (fn [env f args]
                  {:args (update args 0 str/lower-case)})
     :after-fn  (fn [env f args result]
                  {:result (take 10 result)})}))

(def max-results-sym
  (ext/value 'max-results 50
    {:doc "Maximum number of search results returned."}))

(def docs-ext
  (ext/extension
    {:ext/namespace     'com.blockether.vis.ext.documents
     :ext/doc           "Document search and retrieval"
     :ext/version       "1.0.0"
     :ext/author        "Blockether"
     :ext/license       "Apache-2.0"
     :ext/group         "knowledge"
     :ext/subgroup      "documents"
     :ext/ns-alias      {:ns 'vis.ext.docs :alias 'docs}
     :ext/requires      ['com.blockether.vis.ext.editing]
     :ext/prompt        "Document search (use docs/ prefix):
- (docs/search query) — full-text search across documents
- docs/max-results — max results constant (default 50)"
     :ext/activation-fn (fn [env] (seq (list-docs (:db-info env))))
     :ext/nudge-fn      (fn [{:keys [environment iteration prev-expressions]}]
                          (when (and (> iteration 5)
                                    (some :error prev-expressions))
                            "[system_nudge] Document searches are failing."))
     :ext/symbols       [search-sym max-results-sym]
     :ext/classes       {'java.time.LocalDate java.time.LocalDate}
     :ext/imports       {'LocalDate 'java.time.LocalDate}}))

;; Self-register at load time
(ext/register-global! docs-ext)

The LLM sees in the system prompt:

[namespace: docs → vis.ext.docs]
Document search (use docs/ prefix):
- (docs/search query) — full-text search across documents
- docs/max-results — max results constant (default 50)

And calls (docs/search "neural") from :code blocks. Bare (search "neural") does not resolve.

Hook Protocol

Every hook returns a map. Missing keys keep the current value.

Invocation Pipeline

flowchart LR
    Before["before-fn"] --> Fn[":fn"]
    Fn --> After["after-fn"]
    Fn -->|"throws"| OnError["on-error-fn"]
    OnError -->|":result"| Fallback(["Fallback result"])
    OnError -->|":error"| Rethrow(["Re-throw"])
    OnError -->|":fn / :args"| Retry["Retry with new fn/args"]
    Before -->|":result"| ShortCircuit(["Short-circuit"])

wrap-extension wires this up automatically. Direct calls via invoke-symbol-wrapper are rarely needed.

:before-fn — (fn [env f args] → map)

Called before the implementation fn. Can transform inputs or short-circuit.

:after-fn — (fn [env f args result] → map)

Called after the implementation fn returns. Can transform the result.

:on-error-fn — (fn [err env f args] → map)

Called when :fn throws. The return map determines recovery:

If no :on-error-fn is defined, the original exception propagates.

Observability

All hook invocations are logged via taoensso.telemere/log!:

Log data includes :ext (namespace), :sym (symbol name), :phase, and :ms (elapsed milliseconds).

Environment Map

Every callback an extension receives — :ext/activation-fn, :ext/prompt, :ext/nudge-fn, and the symbol hooks (:before-fn, :after-fn, :on-error-fn) — operates on the environment. This is the runtime map that represents one live conversation context.

All Keys

Conversation-scoped (set at create-environment time)

These keys exist on every environment for its entire lifetime:

Query-scoped (added by the query engine per turn)

These keys are assoc’d onto the environment map when a query starts (query/core.clj :: prepare-query-context). They do not exist on the base environment returned by create-environment.

Safe Operations

• Read any key for conditional logic (:db-info, :conversation-id, :sci-ctx, :router, :initial-ns-keys).
• Call persistance.core functions with :db-info for DB reads.
• Bump the var-index cache via bump-var-index! after your tool mutates sandbox state.
• Read sandbox vars via (get-in @(:env (:sci-ctx env)) [:namespaces 'sandbox sym]).

Prohibited Operations

• Close :db-info — the runtime owns the connection lifecycle.
• Swap :extensions directly — use register-extension!.
• Reset :max-iterations-atom or :current-iteration-id-atom — internal iteration-loop state.
• Mutate :sci-ctx namespace maps without calling bump-var-index! — the <var_index> cache will serve stale data.

Nudge System

Nudges are short [system_nudge] strings injected into the iteration prompt to steer the LLM’s behavior. They come from two sources:

1. Built-in nudges — iteration/core.clj (budget warning, var-index overflow, repetition detection)
2. Extension nudges — any extension’s :ext/nudge-fn

Built-in Nudges

When the budget warning fires, the intended agent behavior is explicit: if more work is genuinely needed, call (request-more-iterations n) from :code immediately. Do not ignore the nudge and drift into a weak forced finalize.

Extension Nudges

When an extension provides :ext/nudge-fn, it is called every iteration with a context map. Return a [system_nudge] … string to inject, or nil to skip.

:ext/activation-fn is checked first — if it returns falsy for the current environment, :ext/nudge-fn is not called at all.

Context Map

{:environment            env       ;; the full environment map (see Environment Map)
 :iteration              int       ;; 0-indexed current iteration number
 :current-max-iterations int       ;; live budget cap (includes runtime extensions)
 :prev-expressions       [map]     ;; previous iteration's expressions:
                                   ;;   [{:code str :result any :error str?
                                   ;;     :stdout str :stderr str
                                   ;;     :execution-time-ms int
                                   ;;     :timeout? bool :repaired? bool} …]
                                   ;;   nil on iteration 0 or after error recovery
 :prev-iteration         int       ;; iteration index that produced prev-expressions
                                   ;;   -1 when prev-expressions is nil
 :user-var-count          int}      ;; user-defined vars in the sandbox

Rules

1. Return nil or a non-blank string. Anything else is silently dropped.
2. Prefix with [system_nudge] so the LLM recognises it as system guidance.
3. Keep it short. One line. Nudges >200 chars dilute signal.
4. Never throw. A throwing nudge-fn is caught, logged at :warn, and skipped.
5. Do not mutate environment state. Nudge-fns are observers, not actors.

Example

(ext/extension
  {:ext/namespace 'my-tool
   :ext/doc       "My custom tool"
   :ext/group     "tools"
   :ext/prompt    "Use (my-tool ...) to do X."
   :ext/symbols   [my-tool-sym]
   :ext/nudge-fn  (fn [{:keys [environment iteration prev-expressions]}]
                    (when (and (> iteration 5)
                              (some :timeout? prev-expressions))
                      "[system_nudge] my-tool calls are timing out. Try smaller batch sizes."))})

Pipeline

Inside build-iteration-context (called every iteration):

1. Compute built-in nudges (budget, var-overflow, repetition)
2. Call collect-extension-nudges (iteration/core.clj)
   → for each registered extension with :ext/nudge-fn:
     a. Check :ext/activation-fn against environment
     b. If active, call :ext/nudge-fn with context
     c. Collect non-nil string results
3. Join all nudges with newline
4. Append to iteration context (after <var_index>)