Skip to content

Commit 40dc7e5

Browse files
pete3nclaude
andcommitted
docs: add domain glossary and ADRs for the placeholder round-trip
CONTEXT.md captures the ninjection glossary (injection, parent/child buffer, language header, ninjection block, literal/interpreted placeholder, rename mapping). docs/adr records the three load-bearing decisions: split the parent transform from the injected-keyed header render (0001), Treesitter as the sole structural parser (0002), and the child validates only its own language (0003). TODO.md tracks remaining follow-ups. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent e9c83c5 commit 40dc7e5

6 files changed

Lines changed: 221 additions & 4 deletions

CONTEXT.md

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
# Ninjection
2+
3+
Ninjection is a Treesitter plugin for Neovim that extends Treesitter's
4+
introspection into injected languages: it makes injected-language code blocks
5+
inside a host file (e.g. shell or Lua embedded in Nix) editable as first-class
6+
buffers — with the injected language's LSP, completion, and formatting — then
7+
writes the edits back into the host file. Structural understanding of code is
8+
always Treesitter's job, never hand-rolled string parsing.
9+
10+
## Language
11+
12+
**Injection**:
13+
A region of a host file written in a different language, marked for Treesitter to
14+
parse as that language. The unit ninjection extracts, edits, and writes back.
15+
_Avoid_: snippet, embed, fragment
16+
17+
**Parent buffer**:
18+
The host-language buffer that contains injections (e.g. the Nix file). Modeled as
19+
`NJParent`.
20+
_Avoid_: host buffer, source buffer
21+
22+
**Child buffer**:
23+
The ephemeral buffer holding a single injection's content in the injected
24+
language, where editing happens. Modeled as `NJChild`.
25+
_Avoid_: scratch buffer, edit buffer
26+
27+
**Injected language**:
28+
The language of an injection's content and of the child buffer (e.g. bash).
29+
Distinct from the parent buffer language.
30+
31+
**Language header**:
32+
Ephemeral scaffolding prepended to a child buffer on edit and stripped on
33+
write-back. Supplies what the injected language's LSP needs but the host
34+
synthesizes at build time (e.g. a shell shebang), and carries a delimited block
35+
recording the placeholder substitutions to reverse.
36+
37+
**Ninjection block**:
38+
The delimited, comment-fenced region inside a language header holding real
39+
variable declarations (initialized to the injected language's default value,
40+
e.g. `""` for shell, `nil` for Lua) for the injected placeholders. Satisfies the
41+
injected language's LSP (no undefined-variable diagnostics) and serves as the
42+
round-trip ledger of which placeholders to restore — including rename mappings
43+
for placeholders whose host name is invalid in the injected language. Delimited
44+
in the injected language's comment syntax; tagged with the parent buffer
45+
language.
46+
47+
**Rename mapping**:
48+
A ledger entry recording that an *interpreted* placeholder's host name was
49+
rewritten to an injected-language-safe identifier for editing (e.g. Nix
50+
`pkgs.gnugrep``pkgs_0x2E_gnugrep`, since the dot is invalid in shell — each
51+
invalid character is replaced in place by `_0x<HEX>_`).
52+
Reversed on write-back. Only interpreted placeholders are renamed; literals never
53+
are.
54+
55+
**Literal placeholder**:
56+
A host interpolation that denotes literal text rather than evaluation — in Nix,
57+
`''${var}` produces the literal `${var}`. Ninjection de-escapes it to `${var}`
58+
for editing and re-escapes on write-back. Literals are never renamed: the token
59+
is its own injected-language name, so a literal whose name is invalid in the
60+
injected language is left undeclared, letting that language's LSP surface the
61+
pre-existing error rather than hiding it.
62+
_Avoid_: escaped variable
63+
64+
**Interpreted placeholder**:
65+
A host interpolation the host evaluates — in Nix, `${pkgs.hello}`. Ninjection
66+
rewrites it to an injected-language-safe identifier (see Rename mapping) for
67+
editing and restores it on write-back. Substituting the host's *real* evaluated
68+
value (rather than a default) is a future enhancement.

TODO.md

Lines changed: 15 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,15 @@
1-
# Nix / Shell code implemntation
2-
## Header template
3-
## Literal var placeholders for ''${var} to -> ${var} and back
4-
## Interpereted var placeholders for ${pkgs.var} to ${pkgs_var} and back
1+
# Nix / Shell injected-language editing
2+
3+
Implemented (branch `feat/placeholder-roundtrip`):
4+
- [x] Language header: shebang + fenced ninjection block of real declarations
5+
- [x] Literal var placeholders: `''${var}` <-> `${var}`
6+
- [x] Interpreted var placeholders: `${pkgs.var}` <-> `${pkgs_0x2Evar}`
7+
(in-place `_0x<HEX>` rename, restored via the block's `# <- host` arrows)
8+
9+
Follow-up:
10+
- [ ] checkhealth: report the injected-language Treesitter grammar requirement
11+
for the placeholder round-trip (degrades gracefully today)
12+
- [ ] Resolve the `cfg.debug`-guarded string restorer in `replace()` — currently
13+
inert and accidentally load-bearing (see docs/adr/0001 notes / memory)
14+
- [ ] Real value interpolation for interpreted placeholders (default `""`/`nil` today)
15+
- [ ] Generalise beyond a Nix host / bash+sh injected languages
Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# Split placeholder transform (host-keyed) from header render (injected-keyed)
2+
3+
Making injected code editable spans two independent language axes: rewriting
4+
host interpolations into editable placeholders is governed by the **parent
5+
buffer language's** rules (e.g. Nix decides that `''${x}` is a literal and
6+
`${pkgs.y}` is an interpolation), while building the language header — shebang,
7+
comment delimiter, assignment operator, default value — is governed by the
8+
**injected (child) language's** rules (bash `#`/`=`/`""` vs Lua `--`/` = `/`nil`).
9+
10+
We model these as two stages rather than one function: a parent-keyed *transform*
11+
that returns `(body, ledger)`, then an injected-keyed *header renderer* that turns
12+
the ledger into header text prepended to the child buffer. Restore runs them in
13+
reverse. We chose this over a single function told both languages because it
14+
matches the codebase's existing grain (parent-keyed tables like
15+
`inj_text_modifiers` beside injected-keyed tables like `lsp_map`), keeps each
16+
unit single-responsibility and independently testable, and makes adding an
17+
injected language a matter of supplying a header descriptor rather than editing
18+
the Nix transform.
19+
20+
## Consequences
21+
22+
- The in-buffer ninjection block *is* the ledger, so placeholder data is parsed
23+
back out of the child buffer on restore rather than threaded through
24+
`text_meta`.
25+
- Header height is needed only transiently during `edit()` (to offset the child
26+
cursor by the prepended header) and is passed into `set_cursor`; it is *not*
27+
stored in `text_meta`, and restore locates the header by fence-scan, not a
28+
stored count.
29+
- `text_meta` is therefore not for ninjection's internal bookkeeping. Its purpose
30+
is to be an accessible per-language round-trip channel that a user's modifier
31+
writes and that user's restorer reads, with core shuttling it opaquely — the
32+
extension point for adding language behavior without core changes. It should be
33+
typed opaquely (`table<string, any>`), not narrowed to `table<string, boolean>`.
Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# Treesitter is the sole mechanism for structural introspection
2+
3+
All structural understanding of code — locating injections, distinguishing a Nix
4+
interpolation `${…}` from an escaped literal `''${…}`, identifying language
5+
constructs — is obtained from Treesitter, never from regex or hand-rolled string
6+
parsing. Ninjection is, in effect, a Treesitter plugin that extends Treesitter's
7+
introspection into injected languages; re-implementing slices of a language's
8+
lexer in Lua patterns would be fragile, would silently corrupt users' source,
9+
and is explicitly out of scope to maintain.
10+
11+
## Consequences
12+
13+
- Stage 1 of the placeholder round-trip (see ADR-0001) takes the injected
14+
Treesitter node (or interpolation ranges precomputed from it), not a bare
15+
string, so it can rewrite by node range rather than by pattern match.
16+
- Reverse (write-back) stays in the child buffer: the injected language's TS
17+
locates the variable nodes, and re-adding the parent's literal delimiter is a
18+
trivial prepend onto the TS-identified slice — no parent-language parsing and
19+
no buffer-language switch.
20+
- The ban is on using string processing to *locate or parse* language structure.
21+
Once TS hands us a node range, a simple prepend/substitution on that slice (or
22+
matching ninjection's own fixed `>>> ninjection` fence marker) is not language
23+
parsing and remains acceptable. What is forbidden is scanning or parsing whole
24+
buffers / extracting language elements by hand.
Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# The child buffer validates only its own language
2+
3+
The child buffer is responsible for being valid in the injected language, using
4+
that language's Treesitter grammar and LSP. It is *not* responsible for
5+
validating the text it writes back into the parent buffer. If a user edits a
6+
placeholder into something invalid for the parent language, the parent's own
7+
Treesitter grammar / LSP reports the error after write-back and the user
8+
re-edits there.
9+
10+
We chose this boundary rather than having ninjection re-validate the round-tripped
11+
result against the parent language, because each buffer already has first-class
12+
tooling for its own language and duplicating parent validation inside the child
13+
edit flow adds complexity for no gain — the parent buffer surfaces the error
14+
exactly where the user fixes it.
15+
16+
## Consequences
17+
18+
- Substituting the host's *real* evaluated value into placeholders (real-time
19+
interpolation in the child buffer) is out of scope for now; placeholders are
20+
declared with the injected language's default value (`""`, `nil`).
21+
- The round-trip's correctness obligation is limited to reproducing the user's
22+
intent (escape vs. bare interpolation), not to guaranteeing the result is valid
23+
parent-language code.
24+
- Literals are never renamed. A literal's editable token *is* its
25+
injected-language name, so a literal with an injected-invalid name (`''${cfg.foo}`,
26+
which already emitted invalid shell) is de-escaped but left undeclared — the
27+
injected LSP surfaces the pre-existing error instead of ninjection hiding it
28+
behind a rename. Consequently only interpreted placeholders carry a ledger
29+
arrow, and the reverse rule is: arrow → bare `${host}`; otherwise (bare-in-block
30+
or absent) → escape `''${name}`.
Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# CI shares the local Neovim toolchain from a single source
2+
3+
The e2e test environment is defined once, in `slop-env/` (the Neovim overlay in
4+
`slop-env/nix/` and the runtime config in `slop-env/nvim/`), and consumed through
5+
the root `flake.nix`. CI builds its test image from that same source rather than
6+
from a parallel copy. Concretely, the root flake exposes a headless
7+
`devShells.test` — the same `nvim-dev` derivation, packpath, and
8+
`~/.config/nvim-dev → slop-env/nvim` symlink the local devShell uses, minus the
9+
jail/claude/sandbox machinery — and `ci/Dockerfile` builds from it. The former
10+
duplicate, `ci/nix/kickstart-nix.nvim/`, is deleted.
11+
12+
We adopted this after the test harness diverged between environments: CI ran a
13+
copy pinned to nixpkgs `nixos-25.05` (Neovim 0.11.2) while local development ran
14+
`slop-env/` on `nixos-26.05` (Neovim 0.12.2), and the two config copies had
15+
already drifted (old `require('lspconfig').setup` vs. `vim.lsp.config`/`enable`,
16+
`nixfmt` vs. `nixfmt-rfc-style`). The whole harness depends on one fragile chain
17+
— Plenary spawns each spec in a child Neovim that loads `~/.config/nvim-dev/init.lua`,
18+
and that `init.lua` is what wires `NVIM_PACKPATH` onto the packpath, sets the
19+
indent options that make `lua_ls` emit spaces not tabs, and configures the LSPs
20+
(see [the test-harness notes](../../tests/run.sh)). Two copies of that chain means
21+
two ways for it to silently rot. One source means the version local developers
22+
run is the version CI gates on.
23+
24+
A flake may only import paths inside its own root, so a sub-flake under `ci/nix/`
25+
cannot import `../../../slop-env`. Routing CI through the root flake (which already
26+
does `import ./slop-env/nix/neovim-overlay.nix`) is therefore the mechanism that
27+
makes a single source possible, not merely the tidier option.
28+
29+
## Consequences
30+
31+
- The e2e Docker path is the only CI surface that consolidates here. The
32+
binary-download workflows (`typecheck`, `typecheck-debug`, `gendocs`, `style`,
33+
`lint`) fetch upstream release artifacts and are not Nix-based; they pin their
34+
Neovim / lua-language-server / StyLua versions independently and must be bumped
35+
to match `slop-env` separately to keep the environments aligned. Version parity
36+
across those is a manual obligation, not something the flake enforces.
37+
- `e2e-test.yml` is unchanged: `docker run … nvim-dev --headless -c
38+
"PlenaryBustedDirectory <dir>" -c qa`, with no `minimal_init`. A `minimal_init`
39+
would replace `init.lua` and break the bootstrap chain above, so the harness
40+
must always run the spec child through the real dev config.
41+
- Test fixtures are opened by **relative** path (`tests/ft/…`), never the Docker
42+
WORKDIR absolute (`/ninjection/…`); the working directory is the project root in
43+
both environments.
44+
- The root flake gains a CI-facing output (`devShells.test`). Its inputs include
45+
`jail-nix` / `llm-agents` / `nix-slop-dev`; building `test` must not force those
46+
(they stay lazy). If a build does pull them, the fallback is to make `slop-env/`
47+
its own flake so a lean CI consumer can import it without the agent inputs.
48+
- Bumping the shared source bumps everything at once — Neovim, treesitter
49+
grammars, `lua_ls`, plugins — so a single nixpkgs bump can change formatting and
50+
type-check output together. That is the intended trade: one knob, one
51+
verification pass, no per-environment surprises.

0 commit comments

Comments
 (0)