OKF Vault Kit: five bugs where the premise was wrong

OKF Vault Kit, one of the small tools I publish under CronAI Labs, is an Obsidian vault template that is also an Open Knowledge Format bundle. It is wired to on-device hybrid search through qmd and to a small local model, both of which you install yourself, and the model can search and edit the vault over MCP, the Model Context Protocol. v0.1.0 went out on 2026-09-16. The README calls it an alpha.

I built it in a private repository first, where it reached 0.4.0-alpha.2. The public numbering restarted at v0.1.0, but the opening commit still carries the old label, and its changelog keeps the private record. Its newest entry says an adversarial review plus real runs on macOS and a clean Ubuntu machine found 101 verified defects before the code went public.

Five fixes from that record are below. Three are listed in that newest entry; the code-page and vault-path fixes are listed one build earlier, under 0.4.0-alpha.1. In each one the code did what it said. What was wrong was an assumption about the model, Python's import path, the output encoding, the network, or how the filesystem spells a path.

The smoke test failed while the model worked

The kit's default model, MiniCPM5-2B, reasons before it answers when its thinking toggle is on. kit.py llm smoke asks for a one-line answer, and its max_tokens was sized for that answer. An OpenAI-compatible server takes the chain of thought out of the same pool, so the budget ran out before the answer started. The kit's model notes record the measurement, taken on 2026-09-15 against LM Studio:

The smoke test reported failure. The model was fine. The test was measuring the budget.

The fix adds 512 tokens of headroom to every request, on top of what the caller asked for. The environment variable KIT_LLM_REASONING_TOKENS changes it, and 0 turns it off. On a model that stops on its own this costs nothing, because max_tokens is a ceiling, not a target. The other half of the fix matters as much: an empty answer with finish_reason: length now produces a message saying the model spent its whole budget thinking. Before, it looked like a broken model. A health check that fails falsely is cheaper than one that passes falsely, but it still sends you off to debug the wrong thing. The next one passed falsely.

doctor said the SDK was installed

kit.py doctor reports what is installed. For the MCP SDK it tried import mcp. The kit's own bridge lives in a directory called mcp/, and that directory has no __init__.py.

PEP 420 explains what happens next. Python records a directory without __init__.py and keeps scanning. It creates a namespace package only if no regular package or module turns up anywhere on the path. When kit.py runs, its own directory, the repo root, comes first on sys.path. If the real SDK is installed, its regular package wins further down. If it isn't, the import still succeeds, on the kit's own folder. So the check said "present" on exactly the machines where the SDK was absent.

You can reproduce it with an empty directory (Python 3.11 or newer as python3, no SDK installed):

$ mkdir -p demo/mcp && cd demo
$ python3 -c 'import mcp; print("import mcp: ok, file =", mcp.__file__)'
import mcp: ok, file = None
$ python3 -c 'import importlib.metadata as m; print(m.version("mcp"))'
importlib.metadata.PackageNotFoundError: No package metadata was found for mcp

doctor now asks the package database with importlib.metadata.version("mcp"), and a regression test checks the source for that call.

--help died on Windows

Up to Python 3.14, Python on Windows writes UTF-8 to a real console, and pipes and files get the system's ANSI code page. The sys documentation says so in two sentences that are easy to skip. Python 3.15 switches UTF-8 mode on by default, but the kit supports 3.11 and up.

The changelog says em dashes and arrows "against a legacy console code page" killed --help and doctor with UnicodeEncodeError halfway through their output. It doesn't name the page, and the page decides which character fails. cp1252, the ANSI page a pipe gets on a Western install, has the em dash but no arrows. OEM console pages such as cp437 and cp850 have neither. You can see the difference on any machine:

$ PYTHONIOENCODING=cp1252 python3 -c 'print("doctor — what is installed")' > /dev/null
$ PYTHONIOENCODING=cp1252 python3 -c 'print("doctor → what is installed")' > /dev/null
UnicodeEncodeError: 'charmap' codec can't encode character '\u2192' in position 7: character maps to <undefined>

The fix reconfigures stdin, stdout and stderr to UTF-8 at startup, whatever the page. subprocess calls name their codec too, so note text no longer comes back through a pipe with "—" turned into "�". There is one exception. In stdio mode the MCP bridge leaves stdout alone, because those bytes are JSON-RPC frames that belong to the SDK.

The request to localhost went to the proxy

Managed machines often set HTTP_PROXY and HTTPS_PROXY. Python's urllib sends a request through them unless NO_PROXY names the host, and it makes no exception for loopback. The kit talks to a model server on localhost:1234 by default. The changelog records the result: a running endpoint read as unreachable, and prompt bodies, which carry note content, went to the proxy.

With a throwaway server on 127.0.0.1:1234 (python3 -m http.server 1234 --bind 127.0.0.1 in a second terminal):

$ python3 -c 'import urllib.request as u; print(u.urlopen("http://127.0.0.1:1234/").status)'
200
$ HTTP_PROXY=http://proxy.invalid:8080 python3 -c 'import urllib.request as u; print(u.urlopen("http://127.0.0.1:1234/").status)'
urllib.error.URLError: <urlopen error [Errno 8] nodename nor servname provided, or not known>

The second request never reached port 1234. It went looking for a proxy that doesn't exist. That output is from macOS; Linux words the resolver error differently. On a managed network the proxy does exist, and it gets the request, body and all.

The fix sends every loopback request through an opener with the proxy switched off, and leaves other hosts to the environment. doctor prints a proxy row when one of those variables is set, and when the endpoint is not loopback it says the model calls will go through the proxy and suggests adding the host to NO_PROXY. The kit's test helper probes through the same opener, because with a bare urlopen the whole LLM suite would skip itself on such a machine while LM Studio was running. The tools around the kit don't have the fix, so the proxy guide still tells you to put localhost, 127.0.0.1 and ::1 in NO_PROXY.

The path that was not quite the vault

The bridge hides notes marked sensitivity: confidential from the model. To decide, it maps a note reference to a path relative to the vault, which only works if both sides spell the vault the same way. Often they don't: a symlinked home directory, --vault ., /mnt/c under WSL, or macOS, where /tmp is itself a symlink:

$ ls -ld /tmp
lrwxr-xr-x@ 1 root  wheel  11 Sep 25 04:06 /tmp -> private/tmp

When the mapping failed, it raised an exception, and a blanket except: pass swallowed it. The check carried on as if nothing had happened. The changelog puts the result bluntly: confidential notes were readable whenever the vault path was not canonical.

The fix canonicalises the vault once. A reference that still cannot be mapped comes back as unresolved, and the check treats unresolved as confidential:

def _confidential(self, rel: str) -> bool:
    ...
    if rel == _UNRESOLVED or not self.vault:
        return True
    ...
    try:
        note = kitlib.parse_note(p, self.vault)
    except Exception:  # undecodable or unreadable: we cannot prove it is safe
        return True

Its docstring states the rule: "Every uncertainty answers yes." If "I could not tell" and "it is not confidential" give the same answer, the control is not a control.

The same failure, a control that fails open, showed up four more times in the policy layer before the code went public. The changelog lists them:

• the graph tools ignored confidential hiding
• create_note's template argument could read any file on disk
• the deny and allow lists matched the spelling the model sent rather than the file that would be written
• a confidential note could be declassified with set_property and then read

Three more gaps, closed on 2026-09-16 before the v0.1.0 release, are in Policy, not sandbox.

What I took from it

Three of these were checks that answered a question next to the one they were meant to answer. The smoke test asked whether a budget sized for the answer holds the answer, not whether the model works. doctor asked whether something called mcp imports, not whether the SDK is installed. The confidential check asked whether a path string maps into the vault, not whether the file is confidential. The other two trusted the machine's defaults: the code page a pipe gets, and the proxy a request goes through.

A diff shows what the code does, and much less of what it assumes about the model, the interpreter, the output encoding, the network and the filesystem. A machine that disagrees with those assumptions is the cheapest test of them I know. The habits I kept:

• budget for the model's thinking
• ask the package database, not the importer
• set your own I/O encoding
• keep loopback traffic away from the proxy
• canonicalise both sides before you compare paths
• make "I could not tell" fail closed

Trying it

The kit is MIT-licensed and runs on macOS, Windows (WSL2 is the recommended lane) and Linux. It is designed to run locally, and it ships without containers or scheduling on purpose; Policy, not sandbox explains why. In the words of ALPHA.md, "A 2B model is a 2B model": good at "find this and summarise it", unreliable at reasoning across many notes. If you try it, ALPHA.md says which feedback helps most, and your notes stay a folder of Markdown files whether or not you keep the kit.