diff --git a/docs/book.toml b/docs/book.toml index 093b274..9cc3170 100644 --- a/docs/book.toml +++ b/docs/book.toml @@ -7,6 +7,7 @@ title = "panorama documentation" [output.html] additional-css = ["custom.css"] +mathjax-support = true # [preprocessor.tera] # command = "docs/make-then-tera.sh" diff --git a/docs/src/SUMMARY.md b/docs/src/SUMMARY.md index 4d1a837..708b0f5 100644 --- a/docs/src/SUMMARY.md +++ b/docs/src/SUMMARY.md @@ -13,3 +13,10 @@ # Core Library Developers - [Library API Reference](./lib/api_reference.md) + +# Theory / Notes + +- [RPC Protocol](./notes/rpc.md) +- [Package Ecosystem](./notes/packages.md) +- [Types](./notes/types.md) +- [Data Integrity](./notes/integrity.md) diff --git a/docs/src/index.md b/docs/src/index.md index 3a1787b..50bfc78 100644 --- a/docs/src/index.md +++ b/docs/src/index.md @@ -118,6 +118,8 @@ - Photo viewer + indexing using ML models for face detection / keywords + - Music scrobbler + - Not sure yet: - Automatic sharing of database nodes? This seems like a bad idea for diff --git a/docs/src/notes/integrity.md b/docs/src/notes/integrity.md new file mode 100644 index 0000000..25fab99 --- /dev/null +++ b/docs/src/notes/integrity.md @@ -0,0 +1,16 @@ +# Data Integrity + +## Storage Integrity + +TODO: Look into [Perennial] (code at [perennial-code]) + +[perennial]: https://www.chajed.io/papers/perennial:sosp2019.pdf +[perennial-code]: https://github.com/mit-pdos/perennial + +## Distributed Transactions + +TODO: Look into [two-phase commit] + +[two-phase commit]: https://en.wikipedia.org/wiki/Two-phase_commit_protocol + +TODO: How does this work if not all services in the clusters hold all the nodes? diff --git a/docs/src/notes/packages.md b/docs/src/notes/packages.md new file mode 100644 index 0000000..fbe371b --- /dev/null +++ b/docs/src/notes/packages.md @@ -0,0 +1 @@ +# Package Ecosystem diff --git a/docs/src/notes/rpc.md b/docs/src/notes/rpc.md new file mode 100644 index 0000000..5d65382 --- /dev/null +++ b/docs/src/notes/rpc.md @@ -0,0 +1,25 @@ +# RPC Protocol + +The way that external apps communicate to the database is via an RPC protocol. + +> **Note:** Authentication is not planned at this time. The database should ONLY +> be used to store files that are trusted by all devices. + +Apps declare the RPC protocol by using protobuf to generate Typescript types. +For example: + +```proto +service Todo { + rpc CreateTodo() returns () {} +} +``` + +On the "server"-side, this is converted into types that you can use from the +database code, to actually implement the RPC handlers. On the client-side, this +gives clients a strong foundation to communicate with the database. + +TODO: Look into "compiling" multiple database apps into a single client, +possibly the same way Prisma does? + +TODO: Use a custom language that supports the `sync` primitive to automatically +allow CRDT data types? diff --git a/docs/src/notes/types.md b/docs/src/notes/types.md new file mode 100644 index 0000000..a046953 --- /dev/null +++ b/docs/src/notes/types.md @@ -0,0 +1,24 @@ +# Types + +The panorama database is strongly typed. + +In the database, each metadata key has a corresponding `type` expression, +corresponding to what is allowed in each field. + +TODO: How to handle types from different versions of a package? Semver? + +## Basic Types + +- `number`: As a database value, this is stored as an arbitrary-valued number + with no precision indicator. It's suggested that only numbers that fit in + 64-bit IEEE-754 floating point numbers be stored in here for now, but as the + database spec gets revised this may be further split into integers and + non-integers. + +- `boolean`: This is either true or false. + +- `string`: UTF-8 + +- `bytes`: Binary blobs. + +## Type Constructors