sarthak bhardwaj.
0%
Skip to content
← work

my-lang · side quest · 2024

Writing a tiny language to see what a compiler actually does

A language you can learn in thirty seconds — two keywords, four operators — and a 140-line compiler that turns it into JavaScript. Small on purpose: the point was to watch source text become tokens, a tree and then running code, with nothing hidden in between.

Why build one

A joke language made me realise I'd never looked inside a real one

bhailang went around my feed — a toy language with Hindi slang for keywords. Everyone shared the joke; I got stuck on the machinery. I use a compiler or interpreter every day and had never once watched one turn text into behaviour. The fastest way I know to understand something is to build a bad version of it myself.

The language

Two keywords, four operators, and nothing else

consider declares a variable, print prints one, and expressions can use + - * /. That’s the whole language. The compiler reads a program from a file:

code.txt
consider x = 20
consider y = 20

consider sum = x + y
print sum

Source to running code

Four stages, each one small enough to hold in your head

The my-lang compilation pipelineSourcecode.txtLexerchars → tokensParsertokens → ASTCode genAST → JavaScriptRunnereval in Node
The pipeline. Each stage hands a simpler, more structured thing to the next.
  1. 1
    Lexer. Walks the source one character at a time with a cursor. Whitespace is skipped; runs of letters become a keyword (if they’re consider or print) or an identifier; runs of digits become numbers; + - * / = become operators. Anything else throws “Invalid character” — the language’s only error message.
  2. 2
    Parser. Consumes tokens and builds a tree: a Program whose body holds Declaration and Print nodes. A declaration takes the next token as its name and, after =, everything up to the next keyword as its value.
  3. 3
    Code generator. Walks the tree and emits JavaScript: a declaration becomes let name = value; and a print becomes console.log(name).
  4. 4
    Runner. Reads code.txt with Node’s fs, compiles it, and evaluates the generated JavaScript — with any compile or runtime error caught and reported.
what the compiler produces
let x = 20;
let y = 20;
let sum = x+y;
console.log(sum)

The honest shortcuts

Where the toy cuts corners — and what each one hides

Building it small meant taking shortcuts, and each one turned out to be a signpost to a real compiler problem:

  1. 1
    Expressions are strings, not trees. The parser glues the tokens after = back into text. Operator precedence works only because JavaScript’s does — my language never actually understands x + y * 2.
  2. 2
    It’s a transpiler, not an interpreter. Nothing in my code evaluates anything; it hands JavaScript to eval. That’s why it was quick to build — and why it can run anything you smuggle into an expression.
  3. 3
    Errors know nothing about position. “Invalid character” with no line or column is technically correct and practically useless, which taught me why good compiler errors are their own discipline.

What it taught me

The pipeline is the idea; the language is just the excuse

Where I'd take it

The next three things worth building

  1. 1
    A real expression parser (precedence climbing or Pratt parsing) so + and * mean what the language says, not what JavaScript says.
  2. 2
    A tree-walking evaluator to replace eval — the step that turns a transpiler into an actual interpreter.
  3. 3
    Conditionals, loops and error messages with line and column numbers.

Built with

  • Node.js
  • JavaScript