Crafting Compilers

Everything so far was theory with tiny examples. This chapter is the project: build a complete mini-language end-to-end, one runnable milestone at a time, until “compile hello.dsl and run it” works on your machine.

The Project Shape

The classic recipe (Crafting Interpreters’ path) works as well for a DSL as for a general language: grow the pipeline worst-to-best, always leaving the code runnable.

Milestones That Run

  1. Lex + parse: accept print 1 + 2 and reject print +.
  2. Analyze: reject an undefined name; resolve everything else.
  3. Evaluate: tree-walk print 1 + 2 and print 3 (the Interpreters chapter).
  4. Compile: emit bytecode for the same program (Bytecode & Codegen).
  5. Execute: run the bytecode in a VM (Virtual Machines).
  6. Polish: diagnostics, dumps, and the test corpus.

Keep the Scope Fixed

Decide the feature list on day one (say: numbers, names, print, if, one loop) and freeze it. Every added feature after that is a second project.

Method That Works

Examples First

Write the mini-language’s example programs before any code (the Design Considerations rule). They double as acceptance tests and documentation.

Dump, Don’t Guess

Wire --tokens, --tree, --ir, --bytecode switches from milestone one. When a run fails, one of those dumps shows exactly which stage lies.

Tests That Move With You

Keep a directory of ok/*.dsl (golden output alongside) and bad/*.dsl (expected diagnostic as a comment). The whole suite runs in seconds; run it after every change.

Debugging Compilers

Compiler bugs are unusual because they are invisible until the run: the source looks right, the output is wrong.

Triage by Stage

When print 1 + 2 prints 12: dump the tokens (did the lexer fuse numbers?), dump the tree (did precedence invert?), dump the bytecode (did the emitters reorder?). The dump that looks wrong is the stage on trial.

Minimize Every Failure

Shrink the failing program until it is tiny (remove one feature at a time); the minimal example is usually the diagnosis, and it becomes a permanent regression test.

Example: The Mini-Language

A spec so small the whole project fits this page.

spec.dsl (commented)

# mini.dsl — the frozen feature list, examples first
print 1 + 2                # arithmetic with normal precedence
let price = 5              # names with lexical scope
print price * 2            # output: 10
if price > 3 then print "yes"   # one control structure

Acceptance Rule

The project is done when every line above runs under the tree-walker and under the bytecode VM with identical output — differential testing across the two targets from the Compiler Design chapter.

Next Steps

Continue Phase 2

The two targets you just built get their own chapters: Interpreters for the tree-walker and Virtual Machines & Runtimes for the bytecode engine.

Resources