ytSoftkey VS Code compiler

This is the new, independent VS Code project for the .ytDongle language.

Architecture

• client: VS Code extension entry and user commands
• server: language server and live diagnostics
• Compiler: platform-independent compiler front end
• syntaxes: TextMate syntax highlighting

The current milestone provides tokenization, parsing, semantic type checking, live diagnostics, compile/debug-entry commands, versioned structured IR, flattened bytecode, a checked BIN container, a typed value codec, and a TypeScript reference VM. No UKEY firmware or C runtime code is included in this repository.

The frozen compiler/runtime behavior for globals, reference lifetime, definite assignment, numeric overflow, UTF-8 strings, dynamic arrays and function errors is documented in the source distribution's docs/language-runtime-contract.md.

W8Compile now also writes a typed syntax tree beside future compiler outputs:

build/<source-name>.w8ast.json

The AST preserves complete function bodies, structured primitive/array/struct types, stable structure/member/function/variable IDs, value/ref/out modifiers, resolved user-function calls, array indexes and slices, structure initializers, member access, control flow, and source ranges. It intentionally contains no VM handles, C pointers, IR instructions or BIN offsets.

globalInitializationOrder contains dependency-sorted global variable IDs for the future IR/BIN builder; cyclic initialization is rejected before generation.

Successful and diagnostic compilation results also expose version 1 IR. The IR contains stable IDs, normalized expressions, structured control flow, source ranges and the global initialization order, without C pointers or runtime handles.

The bytecode lowering pass converts structured control flow into labels and jumps. Its execution expressions use append-only numeric opcodes and opcode-specific fixed operand slots. Variable-size values such as call arguments occupy one list slot, so the UKEY runtime never has to discover generic expression fields or dispatch strings. W8BN version 3 stores that program in a compact deterministic payload using stable field IDs, a shared UTF-8 string table, typed tags and variable-length integers, behind a header with version, payload length and checksum. The PC decoder can still open a legacy V1 JSON container only when it carries the current executable schema; V2 and older generic-expression programs must be recompiled. Firmware targets only V3. The TypeScript VM currently executes scalar expressions, variables, globals, calls, conditions, loops, switches and goto, with instruction and call-depth limits.

Bytecode generation also embeds a UKEY ABI v1 resource budget. It uses 32-bit handles and four-byte alignment to calculate the global area, each function's local frame, expression evaluation-stack peak, temporary-value peak, and statically required initial heap. Programs that may grow the heap at runtime are marked explicitly instead of being assigned an unsafe finite estimate. flattenIr can enforce device-specific ceilings and reject an image before W8BN emission.

The VM also supports writable array elements/slices and structure members. Host calls copy value parameters, expose committed ref/out results by parameter index, and restore global state if execution fails. The VS Code W8Compile command writes both .w8ast.json and checked .w8bin files to the configured output directory.

The reference VM enforces device-style limits for array length, UTF-8 string bytes, reachable heap, nested values, elapsed execution time, call depth and total call-stack bytes. Heap checks cover globals, active frames and pending deep copies; failed allocation or API output leaves existing values unchanged. Successful results report peak heap and call-stack usage for PC-side sizing.

Runtime numeric operations enforce each declared integer width. int64 and uint64 execute with exact integer values, checked casts reject out-of-range results, integer division by zero faults, and floating division follows IEEE 754. Reference variables and reference structure members can be rebound to stable cells; nullable array or structure access raises NULL_REFERENCE.

HostCallSession implements the reference host protocol: select an exported function, provide value/ref inputs with setVar, execute call, then retrieve the return value and committed ref/out values. All payloads use the typed binary value codec; failed calls expose no results.

Dynamic built-in APIs are declared in workspace w8apis.json. The compiler validates stable IDs, names, versions and optional signature hashes, then embeds only APIs actually used by the program plus their required permissions. VM API handlers must match the embedded version/signature and may return a deferred commit callback for external side effects.

Source syntax is parsed exactly once by the AST builder. Symbol binding, type checking and control-flow/lifetime checks consume that AST; the retired token-based parser is excluded from the build to prevent grammar drift.

Explicit numeric conversions use C#-style cast syntax:

int32 whole = (int32)12.5;
double restored = (double)whole;
byte low = (byte)(whole + 1);

Numeric types can be explicitly converted in either direction. Arrays and structures cannot be converted to unrelated types, and string/byte conversion is reserved for configurable built-in functions such as stringToBytes.

Reference-like values can be nullable with T?. The first version permits nullable strings, arrays, and structures, but not nullable numeric or boolean values:

string? name = null;
byte[]? payload = null;

Explicit reference variables use the same stable-variable-cell model as ref parameters and structure members. They must be bound when declared and may be rebound with another ref expression:

int32 value = 1;
ref int32 alias = ref value;
alias = 2;
alias = ref value;

Functions may also use local labels and goto. Labels are scoped to one function; duplicate or missing labels are compile-time errors:

goto done;
done:
return;

Function parameters do not support default values. Independent debug inputs are provided by VS Code debug configuration or the future SETVAR call interface:

double calculate(float left, double right) {
    return left + right;
}

Run W8DebugFunction to choose and execute any function in the active source file, including an internal function. The command requests typed JSON inputs, runs the checked bytecode in the reference VM, and prints the return value plus all committed ref/out values. Integer input also accepts quoted decimal text so 64-bit values do not lose precision. Programs that call a configured external API still require a host-provided handler.

Parameters support ref and out, and arrays are valid both as parameters and function return values:

int32[] copyValues(ref int32[] input, out int32[] copied) {
    copied = input;
    return input;
}

Functions are collected before bodies are checked, so forward calls, recursion, and mutual calls are supported. Calls validate argument count, argument types, and ref/out modifiers.

Array operations use value semantics. Assigning one array to another performs a deep copy once the BIN backend is generated. Slices use half-open ranges:

int32 value = source[0];
source[0] = 10;
int32[] copy = source[:];
int32[] middle = source[1:4];
int32[] tail = source[2:];
int32[] head = source[:4];
target[0:3] = source[2:5];

Multidimensional array types and chained indexes are also accepted, for example int32[][] matrix and matrix[0][1].

Structures support value members, ref members, nesting, arrays, function parameters and return values. Structure and array assignment use deep value-copy semantics; a ref member keeps and copies its binding:

struct Address {
    string city;
    ref string street;
}

Address create(ref string road) {
    Address result = Address { city: "上海", street: ref road };
    result.street = "南京路";       // writes road
    return result;
}

All members must be named and initialized. A ref member must use an explicit ref binding. Direct value-recursive structures are rejected; recursion through a ref member or an array is allowed. out is limited to function parameters.

Development

npm install
npm run compile

Open this directory in VS Code and press F5 to launch the Extension Development Host.