TiddlyWiki5 Syntax README

This extension adds TiddlyWiki5 syntax-highlighting and snippets to Visual Studio Code. TiddlyWiki5 is a rich, interactive tool for manipulating complex data, with structures that do not easily fit into conventional tools like spreadsheets or wordprocessors. It is a javascript environment that lives in the Browser and Node.js (and a few other implementations) which builds a Wiki/Content Management System. The design recommends building with "the smallest semantically meaningful chunks" – named tiddlers. A tiddler could be a system javascript module that is loaded on wiki boot, or a few paragraphs of user notes on a topic, or a wikitext function that build a Table of Contents out of a set of related Tags.

When run in the browser, each tiddler is a javascript object in memory (and is also represented as a json in custom <script class="tiddlywiki-tiddler-store" type="application/json"></script> element when the wiki is saved as a "single file" *.html). When run under Node.js, TiddlyWiki saves each wikitext tiddler to disk as a separate text file with a *.tid extension. Other "MIME Types" (images, json text, etc) are saved by the server along with a *.meta file that describes all other tiddler-fields (the file-content becomes the "text" field when loaded into the wiki).

This extension is intended for editing separate tiddler files on disk with Visual Studio Code.

Languages

This extension provides five languages, each with its own grammar and scope.

Each language applies only to the file extensions listed, so adding one changes nothing about how the others behave.

Memetic-Wikitext

*.mem files open as Memetic-Wikitext (text/memetic-wikitext+tiddlywiki), a small extension of TiddlyWiki wikitext used by the Lares agent tooling. Its scope, text.html.tiddlywiki5.memetic-wikitext, includes the wikitext grammar, so ordinary wikitext highlights inside a *.mem file exactly as it does in a *.tid file. On top of that it highlights:

• <<~ name …>> sigils and their closing form <<~/name>>
• the carrier frame's <<^ code="&#x…;" …>> markers: the  marker seats beside the  opener and wraps all the content through its  close, the toml meta fence included
• lar: URIs, read as addresses with their structure — the three-term root, the path, each ?key=value in the query, and the fragment
• ni: digests, the block check a carrier seals its body with
• named parameters written key=value, which TiddlyWiki 5.4 accepts alongside key:value

Every addition uses syntax TiddlyWiki already parses — each of the first two reads as a macro call, and a lar: URI reads as plain text. A *.mem file therefore loads and renders in an unmodified TiddlyWiki without errors, and this extension simply shows more of its structure while editing.

If you do not use *.mem files, nothing here affects you: the language applies to that extension alone.

Features

The colours answer to TiddlyWiki's own parser. Every reading asks what TiddlyWiki builds from the text, and paints that — so a mistake stays where you made it (an unclosed tag, call or inline run stops at the blank line that ends its block, as TiddlyWiki stops it), error colours appear only where TiddlyWiki refuses something, and a pragma, a dash run, a code span or a list item reads the way your wiki will render it.

A tiddler reads in the language its type names. A *.tid or *.meta file parses its field block, then hands the body to the grammar its type: names: wikitext by default; JSON, CSS, JavaScript, HTML, Markdown and plain text as themselves; and image/svg+xml, text/xml and application/xml to the XML grammar. A *.multids file reads the same way from the one type: its header carries, since TiddlyWiki applies that type to every tiddler in the file.

Fenced code reads as real code. A fenced block labelled with a language hands that language to the editor as well as the colourer — 24 of them — so comment toggling, brackets and word selection inside the block follow it. A toml fence reads full TOML 1.1 through a grammar this extension carries privately, since VS Code ships none (ThirdPartyNotices.txt credits it).

Editing meets you halfway. [[ and << close themselves as you type. A double-click takes a TiddlyWiki word, so a system title like $:/core/ui/EditTemplate selects whole. Folding leaves the blank line after a block where it stands.

Your theme reads the structure. Bold and italic carry their font style, not only a colour. A filter reads as its parts — prefix, brackets, operator and operand. A URL's scheme reads as one unit. Struck text dims like an aside in every bundled theme and strikes through wherever the theme supports it. A widget attribute names itself in the scope inspector.

What a line-at-a-time grammar cannot reach, it names. Italic or code carried across a line break and an HTML tag carried across a blank line keep a line or blank-line bound on purpose, so one stray mark cannot repaint every paragraph after it. The ceiling record (editions/tw5-syntax/tiddlers/TextMateCeiling.tid) names each such limit and the kind of reader that could close it.

The grammar grew from these, heavily reworked since:

• https://github.com/shikijs/textmate-grammars-themes/blob/main/packages/tm-grammars/grammars/html.json
• https://github.com/PaulPorfiroff/atom-language-tiddlywiki5
• https://github.com/roma0104/sublime-tid

For contributors

contributing.md carries the detail and SCOPE-NAMES.md carries the rule this repository names its scopes by — what a name promises, the three precedents that settled it, and the three parts of it a gate checks. Every gate answers under two TiddlyWiki readers: the development fork for local work and the pinned 5.4.1 release for CI. LEDGER-3.0.0.md holds the record of every construct, ruling and measurement behind this release. In short, the grammars answer to more than their own tests:

• npm test — assertion files stating what each construct should scope
• npm run test-tools — the tools themselves, which the gates read through
• npm run snap — every sample's whole tokenization, pinned beside it
• npm run canary — an ordinary sentence appended to every sample, which must stay ordinary
• npm run corpus — broad ground, gated on the corpus reaching every declared scope and on nothing bleeding
• npm run test-tools also pins WHICH TiddlyWiki answers: a checkout beside this repository outranks the pinned tiddlywiki devDependency, and TW5_PATH outranks both. Parser work happens in a checkout; the package exists so a contributor holding only this repository runs every gate rather than skipping them
• npm run upstream-coverage -- <path-to-TiddlyWiki5> — TiddlyWiki's own rule regexes, taken to its own tiddlers
• npm run overreach -- --corpus — every scope the grammar paints, handed back to TiddlyWiki's parser: a claim over text it refuses, and a verdict over a construct it builds
• npm run overreach-host — the same question over TiddlyWiki's own tiddlers, against the written rulings
• npm run overreach-cut — the same tiddlers cut short at a seeded offset, so the ground leaves well-formed input behind
• npm run tests-known-gaps — the specimens stating what the grammar does not yet do, which fails when one of them starts passing
• npm run colour-witness — what a reader sees rather than what a scope name says: an opener and its closer read alike in every bundled theme, and a declared distinction reaches enough of them to show
• npm run backtrack-witness — what a half-typed construct costs the tokenizer, over every pattern in every grammar
• npm run attribute-guard — what an attribute-list guard would cost, taken to TiddlyWiki's own tags: how many it would refuse that the parser builds. -- --cut cuts each tag short first, matching the input such a guard would actually meet
• npm run tw5-oracle -- '<wikitext>' — the tree TiddlyWiki builds, and -- --rules the rules it stands
• npm run rule-inventory — every parser rule, the config tiddlers it answers to, and what TiddlyWiki ships for each
• npm run theme-paint -- <scope> — how many bundled themes paint a scope, and -- --families the whole grammar ranked
• npm run gruvbox-report — the six jdinhlife.gruvbox themes, read beside the bundled 65 — reported, never held to a floor. Reaches the installed extension the way grammars.sh reaches VS Code's own bundled grammars (GRUVBOX_THEMES_DIR or VSCODE_EXTENSIONS_DIR name it outright); a machine carrying neither extension nor override prints gruvbox-report SKIP — … and exits 0, which every standalone CI checkout meets honestly
• npm run overreach-corpus-files / overreach-corpus-memetic — the corpus, in both dialects, against the parser and the written rulings
• npm run compose-memes — composition over whatever memetic writing stands beside this checkout; set MEMES to one or more directories
• npm run legibility — whether a reader can tell one construct from another: every pair over every bundled theme, each carrying its own floor
• npm run ceiling — what a TextMate grammar CANNOT reach about TiddlyWiki, measured against the host and naming the kind of reader that closes each one. A ceiling somebody closes RETIRES, and the gate fails until it goes. The bare name READS, so the gate never writes; npm run ceiling:write measures the same and refreshes the harvested TextMateCeiling tiddler with it
• npm run delimiters — what a delimiter inherits from the content it bounds: every contentName in the tree, keyed by shape, each ruled in corpus/delimiter-ledger.txt as a parting that serves a reader or costs one
• npm run package-contents — every path the manifest names, checked inside the built package
• npm run bench — a disposable editor in a container, so you can look at the grammar with your own eyes
• npm run uri-span-measure — a scout instrument: the resolved style of each named span of a sample lar:/ni:/https: line, across every bundled theme, dumped as JSON (> /tmp/uri-measure-<label>.json); npm run uri-span-compare -- before.json after.json diffs two such dumps span by span. Both back the URI-palette rulings cited in LEDGER-3.0.0.md and the memetic-wikitext.json comments; neither is part of the shipped gate set

Colour toggles

A TextMate grammar decides how loudly a construct reads and never which parser rules a wiki stands — nothing in the VS Code API hands a grammar to an extension at runtime. That claim answers to Microsoft's tree; its consequence answers here, and tools/invariants/ships-no-runtime.test.js holds it: no entry point, no activation events, only declarative contributions, no runtime dependency, and nothing executable in the package. A theme paints a scope when one of its own rules names that scope or a dotted prefix of it, which makes the scope name the default. npm run theme-paint -- <scope> measures any scope against the bundled theme set, and -- --families ranks every family this grammar emits.

Two groups repay turning, and both turn from editor.tokenColorCustomizations in your own settings. Nothing else needs a switch: the grammar's structural families already read quiet in every bundled theme.

CamelCase links — off by default

TiddlyWiki has shipped CamelCase linking disabled since 5.3.0 ($:/config/WikiParserRules/Inline/wikilink), so HelloWorld builds no link in a new wiki and the grammar reads it quiet — meta.link.wikilink.tiddlywiki5, which few themes paint where a markup.underline.link. scope inherits nearly every theme's link colour. The construct still carries its own scope, so a wiki that enables CamelCase in Control Panel → Settings colours it back with one rule:

"editor.tokenColorCustomizations": {
  "textMateRules": [
    { "scope": "meta.link.wikilink.tiddlywiki5", "settings": { "fontStyle": "underline" } }
  ]
}

A rule carrying only fontStyle leaves the colour to your theme. A foreground holds one literal colour across every theme you switch to, so give each theme its own block when you want one:

"editor.tokenColorCustomizations": {
  "[Default Dark+]": { "textMateRules": [
    { "scope": "meta.link.wikilink.tiddlywiki5", "settings": { "foreground": "#4fc1ff" } } ] },
  "[Default Light+]": { "textMateRules": [
    { "scope": "meta.link.wikilink.tiddlywiki5", "settings": { "foreground": "#0451a5" } } ] }
}

The suppressing ~ keeps its punctuation colour either way: wikilinkprefix carries its own rule, TiddlyWiki ships it enabled, and it consumes the ~ whether or not CamelCase stands.

Constructs your theme leaves quiet

No construct reads as body text in more than a quarter of the 65 bundled themes, so this grammar asks a reader for no fontStyle rule of their own. A construct that starts reading quiet joins a table here the day it does.

Generated by npm run reading-recipe from corpus/prose-reading-ledger.txt.

Verdicts — on by default

A verdict answers to what TiddlyWiki refuses, and never to what another language retired. TiddlyWiki parses any tag name, any attribute name, a lone angle bracket and a stray & into the page, so none of them draws one. A single verdict stands: invalid.illegal.field.name, on a field name a *.tid or *.meta header cannot hold. Most themes paint it as an error — the two VS Code ships among them — and MIGRATION.md records the names that went.

Name the family, never a member. A theme selector reaches a scope by dot-bounded prefix, so one rule on invalid.illegal reaches that verdict and keeps reaching it when a name gains a segment. To read it quietly:

"editor.tokenColorCustomizations": {
  "textMateRules": [
    { "scope": "invalid.illegal", "settings": { "foreground": "#808080" } }
  ]
}

That selector also reaches an embedded block's own verdicts — a CSS or JavaScript region inside a tiddler — which is usually what a reader quieting errors wants. npm run rule-inventory prints the exact names, so you never need this page to list them.

Both blocks work per workspace folder in .vscode/settings.json, so one wiki can answer differently from another.

Why this extension ships no colour of its own

An extension may contribute editor.tokenColorCustomizations through contributes.configurationDefaults. VS Code registers it, warns nothing, and it takes effect; some 450 published manifests do exactly that. This one declines, for three reasons that hold whatever the colour.

No hardcoded colour survives an arbitrary theme. A foreground that reads well on Default Dark+ reads as noise on a light, high-contrast or pastel theme, and the extension never learns which one you run. Theming belongs to the theme and to you.

A contributed default cannot be scoped to a language. Only settings marked language-overridable reach the per-language schema, and editor.tokenColorCustomizations carries no such mark — VS Code's theme maintainers have closed that request directly. So a colour contributed here would apply in every file you open, not only in wikitext. The one containment mechanism left is the language suffix every scope name carries: each grammar closes its names on its own — .tiddlywiki5, .memetic-wikitext, .tid-file, .multids-file, .fields — and tools/invariants/scope-suffix.test.js holds every one to it.

A contributed default merges INTO your settings rather than sitting under them. VS Code deep-merges object defaults, so writing your own editor.tokenColorCustomizations would not clear the extension's rules — removing them would take an explicit empty "textMateRules": [], which would destroy your own alongside. An opinion you cannot turn off without losing your own work is not a default.

So the two blocks above stay a snippet you paste, theme-scoped, under your hand. What this extension does contribute by default is behavioural and carries no colour: files.associations, mapping each extension to its language. VS Code's own 24 built-in extensions that use configurationDefaults set no colour either.

Per-rule switching — one toggle for each of TiddlyWiki's parser rules, resolved down a bag stack — waits on the tree-sitter and language-server work, where a live parser can answer for the file in front of it. npm run rule-inventory reports that surface today.

Known Issues

Please report issues or offer Pull Requests at the GitHub Repository:

• https://github.com/joshuafontany/VSCode-TW5-Syntax

Release Notes

• CHANGELOG.md — what each release grants you
• LEDGER-3.0.0.md — every construct, ruling and measurement behind 3.0.0
• MIGRATION.md — every scope name 3.0.0 retired, with what it stands as now

Thank You for trying TiddlyWiki5 Syntax Highlighting for Visual Studio Code

Enjoy!

• Joshua Fontany