---
title: Compiler from scratch
date: 2023-09-08T06:17:00.840Z
tags:
  - programming-languages
draft: true
toc: true
---

import CodeBlock from "./CodeBlock.astro";
import Playground from "./Playground.astro";

Just for fun, let's write a compiler that targets WebAssembly.
I'm writing this post as I'm discovering how this works, so join me on my journey!

## Producing a working "binary"

I don't know how WebAssembly actually works, so here's some of the resources I'm
consulting:

- https://developer.mozilla.org/en-US/docs/WebAssembly/Using_the_JavaScript_API
- https://webassembly.github.io/spec/core/index.html

A compiler for a general language is quite an undertaking, so let's start with
the proverbial "Hello, world" program, just to write some output to the screen.
This ...isn't very clear either. First of all, how do we even get output from
WebAssembly?

Well, it looks like according to [this][exported-functions] document, you can
essentially mark certain wasm concepts as "exported", and access them from
`obj.instance.exports`. Let's start by trying to export a single number.

[exported-functions]: https://developer.mozilla.org/en-US/docs/WebAssembly/Exported_functions

### Returning a number from WebAssembly

We can use tables to export a number from wasm to JavaScript so we can access it
and print it to the screen. Based on the [MDN example], we can tell that we'll
need to be able to export modules, functions, and tables. We can use the [binary
format spec] to figure out how to produce this info.

[mdn example]: https://github.com/mdn/webassembly-examples/blob/5a2dd7ca5c82d19ae9dd25d170e7ef5e9f23fbb7/js-api-examples/table.wat
[binary format spec]: https://webassembly.github.io/spec/core/binary/index.html

Starting off, a class that we can start writing binary data to:

<CodeBlock
  code={`
    function WasmWriter(size) {
      this.buffer = new ArrayBuffer(size ?? 1024);
      this.cursor = 0;
    }
    
    // Helper function for displaying the number of bytes written as an array
    WasmWriter.prototype.asArray = function() { return [...new Uint8Array(this.buffer.slice(0, this.cursor))]; };
    WasmWriter.prototype.display = function() { return "[" + this.asArray().map(x => x.toString(16).padStart(2, '0')).join(", ") + "]"; };
  `}
/>

<Playground runAction={`displayResult(new WasmWriter().display());`} />

We'll want to write some stuff into it. Like bytes?

<CodeBlock
  code={`
    WasmWriter.prototype.write = function(obj) {
      const len = obj.len?.();
      const view = new Uint8Array(this.buffer);
      obj.write({ 
        recurse: (obj) => this.write(obj),
        emit: (byte) => { view[this.cursor] = byte; this.cursor += 1 }
      });
    };
`}
/>

Or [integers][int spec]? Let's use the [algorithm given on Wikipedia][uleb algo]
here.

[int spec]: https://webassembly.github.io/spec/core/binary/values.html#integers
[uleb algo]: https://en.wikipedia.org/wiki/LEB128#Unsigned_LEB128

<CodeBlock
  resultName="UInt"
  code={`
    class UInt {
      constructor(num) { this.num = num; }
      write({ emit }) {
        let num = this.num;
        if (num === 0) { emit(0); return }
        do {
          let byte = num % 128;
          num = num >> 7;
          if (num !== 0) byte = byte | 128;
          emit(byte);
        } while(num !== 0);
      };
    }
`}
/>

<Playground
  label="Encode some ints"
  runAction={`
    let ints = [10, 100, 1000, 10000, 100_000];
    displayResult(ints.map(x => {
      const writer = new WasmWriter();
      writer.write(new UInt(x));
      return \`\${x} encodes to \${writer.display()}\`;
    }).join("\\n"));
  `}
/>

Perfect. What do we still need to encode a complete WebAssembly program? Reading
[this][binary modules spec], I guess we'll need functions, tables, and modules.
Let's keep going, starting with [functions][func type spec].

[binary modules spec]: https://webassembly.github.io/spec/core/binary/modules.html#binary-module
[func type spec]: https://webassembly.github.io/spec/core/binary/types.html#binary-functype

<CodeBlock
  resultName={["Vec", "ResultType", "NumType", "FuncType"]}
  code={`
    class Vec {
      constructor(items) { this.items = items; }
      write({ recurse }) {
        recurse(new UInt(this.items.length));
        this.items.forEach(item => recurse(item));
      }
    }

    class ResultType {
      constructor(valTypes) { this.valTypes = valTypes; }
      write({ recurse }) { recurse(new Vec(this.valTypes)); }
    }

    class NumType {
      constructor(type) { this.type = type; }
      write({ emit }) {
        emit({ "i32": 0x7f, "i64": 0x7e,
                "f32": 0x7d, "f64": 0x7c }[this.type]);
      };
    }

    class FuncType {
      constructor(rt1, rt2) { this.rt1 = rt1; this.rt2 = rt2; }
      write({ emit, recurse }) { emit(0x60); recurse(this.rt1); recurse(this.rt2); };
    }

`}
/>

If you run this, you'll see that it prints out what we expected:

- `0x60` designates that it's a function type
- `0x00` means the list of parameter types is empty
- `0x01` means the list of return types has 1 item
  - that item is `0x7f`, corresponding to `i32`

<Playground
  label="Encode [] -> [i32] function"
  runAction={`
    const writer = new WasmWriter();
    writer.write(new FuncType(
      new ResultType([]),
      new ResultType([new NumType("i32")]),
    ));
    displayResult(writer.display());
  `}
/>

Now, on to [tables][table type spec]:

[table type spec]: https://webassembly.github.io/spec/core/binary/types.html#table-types

<CodeBlock
  resultName={["TableType", "RefType", "Limit"]}
  code={`
    class TableType {
      constructor(et, lim) { this.et = et; this.lim = lim; }
      write({ recurse }) { recurse(this.et); recurse(this.lim); }
    }

    class RefType {
      constructor(type) { this.type = type; }
      write({ emit }) { emit({"func": 0x70, "extern": 0x6f}[this.type]) }
    }

    class Limit {
      constructor(min, max) { this.min = min; this.max = max; }
      write({ emit, recurse }) {
        const min = new UInt(this.min), max = new UInt(this.max);
        if (this.max) { emit(0x1); recurse(min); recurse(max); }
        if (this.max) { emit(0x0); recurse(min); }
      }
    }

`}
/>

<Playground
  label="Encode a table of functions with limit [1, 5]"
  runAction={`
    const writer = new WasmWriter();
    writer.write(new TableType(
      new RefType("func"),
      new Limit(1, 5),
    ));
    displayResult(writer.display());
  `}
/>

#### Module

Ok, let's put this all together and make a [module][module spec]!

[module spec]: https://webassembly.github.io/spec/core/binary/modules.html#binary-module

<CodeBlock
  resultName={["Module"]}
  code={`
    class Module {
      constructor(sections) { this.sections = sections; }
      write({ emit, recurse }) {
        emit(0x00); emit(0x61); emit(0x73); emit(0x6d);
        emit(0x01); emit(0x00); emit(0x00); emit(0x00);
        this.sections.map(section => recurse(section));
      }
    }
`}
/>

<Playground
  label="Encode a module!!"
  runAction={`
    const writer = new WasmWriter();
    writer.write(new Module([
      new FuncType(new ResultType([]), new ResultType([new NumType("i32")])),
      new TableType(new RefType("func"), new Limit(0)),
    ]));
    displayResult(writer.display());

`}
/>
