Skip to content

examples.md

Latest commit

 

History

History
525 lines (401 loc) · 20.7 KB

examples.md

File metadata and controls

525 lines (401 loc) · 20.7 KB
Raw

LangForge Examples

Document id: lang-forge-examples-v1

Status: active

Last updated: 2026-07-10

Owner: Project maintainers

Scope: Examples and fixture guide for LangForge

How To Read The Examples

Each runnable example has the same broad shape:

*.lf grammar file
  -> lang-forge generate
  -> generated scanner/parser under generated/ or Generated/
  -> handwritten reducer or adapter outside generated/
  -> command-line demo under cmd/
  -> binary and logs under dist/

Only generated/ or Generated/ is produced by LangForge. Directories such as cmd, semantics, .NET project files, and adapter files outside generated folders are normal source code. They are written by the example author and may import the generated package.

Several Go files start with //go:build langforge_generated. That is a Go build tag. It tells Go to compile those files only after the example Makefile has generated the local scanner/parser package and passes -tags langforge_generated.

For a fuller beginner guide, including what {go: add} means, read Generated Code And Semantics. For reusable per-language parser facade, reducer, library, dependency injection, and multi-parser layouts, read Handwritten Integration Guide.

The examples accept a LANG_FORGE Makefile override. That command can be the source checkout, a standalone binary, an installed binary, or a Docker image. For the reusable Makefile and Docker patterns, read Invocation And Layout Patterns.

Examples also default LANG_FORGE_VERBOSITY to 1, so make run, make test, and make generate show the major LangForge stages while the specs are validated and generated. Set it to 0 for quiet script output, or raise it while debugging a grammar:

make -C examples/go/calc run LANG_FORGE_VERBOSITY=0
make -C examples/go/calc generate LANG_FORGE_VERBOSITY=2
make -C examples/parser-algorithms test LANG_FORGE_VERBOSITY=3

make examples-parity runs two fast source checks. The spec parity checker compares equivalent .lf files after normalizing target-specific directives. The action-manifest parity checker builds the same contracts written to langforge.actions.json and compares action labels, RHS labels, portable typed semantic roles, typed/untyped status, and recovery reporting contracts. When a target-specific difference is intentional, document it in examples/manifest-parity.allowlist.json with the reported family, target, path, and a short reason.

Example Requirements

The Go examples need Go and make. The C# examples need the .NET 10.0 SDK because their projects target net10.0. The C examples need GCC or another C11-capable compiler for build/run targets; set CC=clang or another compiler when needed. The C++ examples need g++, clang++, or another C++17-capable compiler; set CXX=clang++ or another compiler when needed. The full matrix is in Requirements.

Calc Combined Spec

The main example is:

It defines:

  • a number/operator lexer;
  • an expression grammar with precedence encoded through nonterminals;
  • Go package name calc;
  • generated output under examples/go/calc/generated, created on demand.

Run:

make -C examples/go/calc run

The runnable demo reads examples/go/calc/input.calc, regenerates the scanner/parser, builds dist/calc-demo, parses through NewReaderScanner, evaluates the expression through generated reduction hooks, and writes the same report to examples/go/calc/dist/calc-demo.log. It also tokenizes afterward to print a teaching token stream in the report.

The calc family is the compact cross-target demonstration of streamed scanner input. Go uses io.Reader, C# uses TextReader/Stream, C uses a read callback with *_stream_scanner, and C++ uses InputStreamScanner over std::istream. Token-list parsing remains in the examples for tests, debugging, tests, and token inspection.

The calc example also demonstrates reducer-mode handwritten package references: calc.lf declares a %semantic go import and the demo wires that package as the generated parser reducer.

In calc, {go: add} and {go: subtract} are labels. They are not arithmetic code generated by LangForge. The handwritten reducer in examples/go/calc/semantics/reducer.go uses generated semantic action constants and ReducerMap to map those labels to arithmetic behavior.

The sample expression is:

1 + 2 * (3 - 4.5)

The example also supports direct generated-code testing:

make -C examples/go/calc test

Generated artifacts under examples/go/calc/generated and examples/go/calc/dist are ignored by Git. Use make -C examples/go/calc clean to remove them.

C# Mirror Examples

C# examples live under:

They mirror the Go grammar shapes but use %target csharp, C# namespaces, and {csharp: ...} action labels:

Expr : Expr Plus Term {csharp: add}

Run the C# family:

make -C examples/csharp/calc run
make -C examples/csharp/datakeeper run
make -C examples/csharp/draw run
make -C examples/csharp/parser-recovery run
make -C examples/csharp/vehicle-report run

Test:

make -C examples/csharp/calc test
make -C examples/csharp/datakeeper test
make -C examples/csharp/draw test
make -C examples/csharp/parser-recovery test
make -C examples/csharp/vehicle-report test

Each Makefile validates the spec, generates C# scanner/parser files under Generated/ with .g.cs filenames, builds with .NET 10, runs a handwritten C# reducer, and writes a local report under dist/.

The C# DRAW example writes dist/sample-csharp.png and a render report. It is the C# mirror of the Go DRAW PNG workflow, with handwritten C# AST and PNG code outside Generated/.

For a reusable compiler-style C# starter, see examples/templates/csharp/layered-compiler. It hides generated parser details behind IMiniCompilerParser, returns a domain ParseResult<ProgramNode>, maps generated typed reducer contexts in Semantics/, and keeps Program.cs as a thin demo.

C++ Examples

The C++ examples live under:

They use %target cpp, C++ namespace packages, and {cpp: ...} action labels:

Expr : Expr Plus Term {cpp: add}

Run it:

make -C examples/cpp/calc run
make -C examples/cpp/datakeeper run
make -C examples/cpp/draw run
make -C examples/cpp/parser-recovery run
make -C examples/cpp/vehicle-report run

Test it:

make -C examples/cpp/calc test
make -C examples/cpp/datakeeper test
make -C examples/cpp/draw test
make -C examples/cpp/parser-recovery test
make -C examples/cpp/vehicle-report test

The handwritten main.cpp files use generated SemanticAction enum values and ReducerMap instead of embedding semantics in generated parser code. That mirrors the preferred C++ backend style from ADR-0014: generated tables stay static and deterministic, while handwritten semantics are ordinary C++ functions and lambdas keyed by action IDs.

For a modern C++ starter that hides generated APIs behind a domain parser facade, uses direct typed reducer handlers, documents ownership with std::unique_ptr and std::variant, and includes CMake integration, see examples/templates/cpp/layered-compiler.

C Mirror Examples

C examples live under:

They mirror the same language scenarios with %target c, C-oriented %package prefixes, and {c: ...} action labels. LangForge generates tokens.h, scanner.h, scanner.c, parser.h, and parser.c under each example's generated/ directory.

Run the C family:

make -C examples/c/calc run
make -C examples/c/datakeeper run
make -C examples/c/draw run
make -C examples/c/parser-recovery run
make -C examples/c/vehicle-report run

Test:

make -C examples/c/calc test
make -C examples/c/datakeeper test
make -C examples/c/draw test
make -C examples/c/parser-recovery test
make -C examples/c/vehicle-report test

The C Makefiles validate and generate without requiring a C compiler. Build and run steps are skipped with a clear message if CC is unavailable. When a C compiler is present, the examples compile the generated sources together with handwritten main.c reducers and the shared examples/c/common helper module.

The handwritten C examples include generated/parser.h directly instead of relying on the Makefile include path to find parser.h. The generated header is still the single source of truth for parser/scanner types; the explicit path just makes IDE code navigation work after make -C examples/c/... generate.

The C DRAW example writes an actual PNG file through a handwritten C AST and interpreter. It mirrors the Go/C# DRAW flow while using a tiny local RGB/PNG helper instead of external image libraries.

Learning Progression

The examples are meant to be read in increasing complexity:

Step Example What it teaches
0 examples/templates/*/mini-compiler Copyable scanner/parser/compiler starter shape with AST, reducer, stack-code lowering, runtime, diagnostics, and tests
1 examples/templates/*/library-dsl Reusable library architecture with domain model, typed reducer, parser facade, diagnostics formatter, thin demo, and smoke tests
2 examples/templates/csharp/layered-compiler Modern C# layered compiler architecture with Ast/, Semantics/, Parsing/, IMiniCompilerParser, domain results, and DI-friendly semantic policies
3 examples/templates/cpp/layered-compiler Modern C++17 layered compiler architecture with public headers, source-based parsing, direct typed reducers, move-only AST ownership, and CMake
4 examples/go/calc Small combined .lf, generated scanner/parser, reducer-backed expression value, simple CLI report
5 examples/{go,csharp,c,cpp}/parser-recovery Reserved error productions, synchronization, expected-token aliases, multiple diagnostics, result cleanup/inspection, and progress guarantees
6 examples/parser-algorithms LR parser algorithm differences and expected SLR conflict
7 examples/go/datakeeper DSL syntax, generated reduction hooks, AST adapter, stack-machine lowering, mock execution
8 examples/go/draw DSL syntax, generated reduction hooks, interpreter, reusable blocks, rendered output
9 examples/go/vehicle-report Exercise-style Flex/Bison migration shape, generated reductions, AST, XML-like report
10 examples/csharp/* The same example set generated for C# with .g.cs output, reducer enums, .NET build/run checks, and mock reports
11 examples/c/* The same example set generated for C with conventional .h/.c files, reducer function pointers, and C-friendly reports/artifacts
12 examples/cpp/* The same example set generated for C++17 with enum class actions, ReducerMap, and static table lookup
13 examples/benchmarks Optional scanner/parser performance examples for throughput, allocation, source parsing, lexeme slices, reducer dispatch, and recovery overhead
14 testdata/ucdt Legacy split .l/.y inspiration fixtures and regression checks

For starter-project guidance rather than a demo tour, read Example Template Guide. It explains the examples/templates folders, shared fixtures under examples/testdata, typed reducer helpers, parser facade templates, and reusable fragments under examples/mk.

See Learning Path for the guided route through these examples.

Optional Benchmarks

Benchmark examples live under examples/benchmarks. They are not part of make examples-test or make ci.

Run the Go benchmark suite:

make examples-benchmarks
make examples-benchmarks-go BENCH_COUNT=5 BENCH_TIME=2s
make examples-benchmarks-csharp CSHARP_BENCH_FILTER='*CalcParse*'
make examples-benchmarks-csharp CSHARP_BENCH_JOB=medium CSHARP_BENCH_FILTER='*CalcParse*'
make examples-benchmarks-report BENCH_COUNT=10 BENCH_TIME=1s

The Go suite uses go test -bench and -benchmem. The C# suite uses BenchmarkDotNet with memory diagnostics and writes artifacts under dist/benchmarks/csharp. Both use the same vocabulary: ParseFromLexemeSource includes scanner/lexeme-source work, while ParsePreTokenized parses tokens prepared before the timed loop. The default path is quick mode. Use repeated Go runs through BENCH_COUNT=5 or 10, and use CSHARP_BENCH_JOB=medium or default, before drawing before/after conclusions. BenchmarkDotNet Error can be large in short mode because it intentionally uses few iterations.

The benchmark report target writes raw and summarized output under dist/benchmarks, including go-benchmarks-summary.md and csharp-benchmarks-summary.md. Summary paths are repository-relative where practical so reports can be shared between local checkouts.

Static generated artifact metrics are written by make examples-benchmarks-report as Markdown and JSON under dist/benchmarks; they are not reported as timed benchmark rows.

C and C++ benchmark harnesses remain future optional work. Their examples already expose the same source/token and typed/boxed entry points, but target-specific timing harnesses should stay outside normal CI.

UCDT Calc Fixture

Curated copies of the Pascal UCDT calc input files live under:

Run:

go run ./cmd/lang-forge validate \
  --lex testdata/ucdt/calc/calc.l \
  --yacc testdata/ucdt/calc/calc.y

This fixture is used to keep split-file parsing and sample translation honest while the .lf format evolves. It is not a contract with UCDT.

Additional curated UCDT-derived fixtures live under testdata/ucdt/draw and testdata/ucdt/metas. See UCDT reference for the source role and validation evidence.

Parser Algorithm Fixtures

Small LR parser algorithm fixtures live under:

They demonstrate one grammar that validates under LALR(1), IELR(1), and canonical LR(1) but reports an expected conflict under SLR, plus another grammar where LALR reports an expected false merge conflict while IELR(1) and canonical LR(1) validate. Run:

make -C examples/parser-algorithms test

See Parser Algorithms for the detailed automata explanation, pseudo-code, and algorithm selection guidance.

DataKeeper Scripting Compiler Demo

The DataKeeper scripting demo lives under:

It reconstructs the small Irony-based DataKeeperScripting language as a real LangForge example. The syntax is defined in examples/go/datakeeper/datakeeper.lf, the generated Go scanner/parser is created on demand under examples/go/datakeeper/generated, and the reducer-backed semantic layer lowers recognized scripts into stack-machine code.

The action labels in datakeeper.lf build an AST through handwritten adapter code. LangForge recognizes the syntax and reports reductions; the example code decides how those reductions become script statements and VM instructions.

datakeeper.lf -> generated scanner/parser reducer -> AST -> stack-machine instructions -> mock execution report

Run:

make -C examples/go/datakeeper run

The demo reads examples/go/datakeeper/sample.dks, builds dist/datakeeper-demo, and writes a report to examples/go/datakeeper/dist/datakeeper-demo.log. It does not call a real database. It logs mocked RunSQL, AddObject, RemoveObject, and RunObjectsJob adapter calls, plus the VM stack trace.

To use a standalone LangForge binary instead of running from source:

make build
make -C examples/go/calc LANG_FORGE=../../../dist/lang-forge run
make -C examples/go/datakeeper LANG_FORGE=../../../dist/lang-forge run
make -C examples/go/draw LANG_FORGE=../../../dist/lang-forge run
make -C examples/go/vehicle-report LANG_FORGE=../../../dist/lang-forge run
make -C examples/csharp/calc LANG_FORGE=../../../dist/lang-forge run
make -C examples/csharp/datakeeper LANG_FORGE=../../../dist/lang-forge run
make -C examples/csharp/draw LANG_FORGE=../../../dist/lang-forge run
make -C examples/csharp/vehicle-report LANG_FORGE=../../../dist/lang-forge run
make -C examples/c/calc LANG_FORGE=../../../dist/lang-forge run
make -C examples/c/datakeeper LANG_FORGE=../../../dist/lang-forge run
make -C examples/c/draw LANG_FORGE=../../../dist/lang-forge run
make -C examples/c/vehicle-report LANG_FORGE=../../../dist/lang-forge run
make -C examples/cpp/calc LANG_FORGE=../../../dist/lang-forge run
make -C examples/cpp/datakeeper LANG_FORGE=../../../dist/lang-forge run
make -C examples/cpp/draw LANG_FORGE=../../../dist/lang-forge run
make -C examples/cpp/vehicle-report LANG_FORGE=../../../dist/lang-forge run

DRAW PNG Renderer Demo

The DRAW demo lives under:

It adapts the UCDT SAMPLES/DRAW language idea as a LangForge example that renders real PNG images. The syntax is defined in examples/go/draw/draw.lf, the generated Go scanner/parser is created on demand under examples/go/draw/generated, and the reducer-backed semantic layer builds an AST before the interpreter handles variables, math functions, reusable figure blocks, draw, repdraw, and drawing primitives.

The DRAW adapter is handwritten source code. It consumes generated reductions, builds drawing nodes, and leaves image rendering to ordinary Go package code outside generated.

DRAW is also the larger typed-semantics example. Its four target specs share labels such as width=Expr, target=FigureReference, and right=Term, plus target-native nonterminal result declarations. Supported targets generate typed contexts or adapters for DRAW actions; the handwritten adapters use named fields directly or validate typed contexts before delegating to boxed reducers. The Go dependency-only model package prevents an import cycle between generated parser code and the application AST.

draw.lf -> generated scanner/parser reducer -> AST -> interpreter -> PNG

Run:

make -C examples/go/draw run

The demo reads examples/go/draw/sample.draw, builds dist/draw-demo, renders examples/go/draw/dist/sample.png, and writes a report to examples/go/draw/dist/draw-demo.log.

The sample language includes canvas and style commands:

canvas 960,640;
background #101820;
stroke #F2AA4C;
fill none;
width 2;

Vehicle Report Demo

The vehicle report demo lives under:

It is inspired by a small Flex/Bison-style compiler exercise that parsed a car = { ... } source file and printed XML-like output. The LangForge version keeps that migration-friendly shape while using a combined .lf grammar, generated Go scanner/parser output, and reducer-backed AST construction.

vehicle.lf -> generated scanner/parser reducer -> AST -> text/XML-like report

Run:

make -C examples/go/vehicle-report run

The demo reads examples/go/vehicle-report/sample.vehicle, builds dist/vehicle-report-demo, prints the report, and writes examples/go/vehicle-report/dist/vehicle-report-demo.log.