notes

docs/book.toml

@@ -7,6 +7,7 @@ title = "panorama documentation"
 
 [output.html]
 additional-css = ["custom.css"]
+mathjax-support = true
 
 # [preprocessor.tera]
 # command = "docs/make-then-tera.sh"

docs/src/SUMMARY.md

@@ -13,3 +13,10 @@
 # Core Library Developers
 
 - [Library API Reference](./lib/api_reference.md)
+
+# Theory / Notes
+
+- [RPC Protocol](./notes/rpc.md)
+- [Package Ecosystem](./notes/packages.md)
+- [Types](./notes/types.md)
+- [Data Integrity](./notes/integrity.md)

docs/src/index.md

@@ -118,6 +118,8 @@
 
   - Photo viewer + indexing using ML models for face detection / keywords
 
+  - Music scrobbler
+
   - Not sure yet:
 
     - Automatic sharing of database nodes? This seems like a bad idea for

docs/src/notes/integrity.md

@@ -0,0 +1,16 @@
+# Data Integrity
+
+## Storage Integrity
+
+TODO: Look into [Perennial] (code at [perennial-code])
+
+[perennial]: https://www.chajed.io/papers/perennial:sosp2019.pdf
+[perennial-code]: https://github.com/mit-pdos/perennial
+
+## Distributed Transactions
+
+TODO: Look into [two-phase commit]
+
+[two-phase commit]: https://en.wikipedia.org/wiki/Two-phase_commit_protocol
+
+TODO: How does this work if not all services in the clusters hold all the nodes?

docs/src/notes/packages.md

@@ -0,0 +1 @@
+# Package Ecosystem

docs/src/notes/rpc.md

@@ -0,0 +1,25 @@
+# RPC Protocol
+
+The way that external apps communicate to the database is via an RPC protocol.
+
+> **Note:** Authentication is not planned at this time. The database should ONLY
+> be used to store files that are trusted by all devices.
+
+Apps declare the RPC protocol by using protobuf to generate Typescript types.
+For example:
+
+```proto
+service Todo {
+  rpc CreateTodo() returns () {}
+}
+```
+
+On the "server"-side, this is converted into types that you can use from the
+database code, to actually implement the RPC handlers. On the client-side, this
+gives clients a strong foundation to communicate with the database.
+
+TODO: Look into "compiling" multiple database apps into a single client,
+possibly the same way Prisma does?
+
+TODO: Use a custom language that supports the `sync` primitive to automatically
+allow CRDT data types?

docs/src/notes/types.md

@@ -0,0 +1,24 @@
+# Types
+
+The panorama database is strongly typed.
+
+In the database, each metadata key has a corresponding `type` expression,
+corresponding to what is allowed in each field.
+
+TODO: How to handle types from different versions of a package? Semver?
+
+## Basic Types
+
+- `number`: As a database value, this is stored as an arbitrary-valued number
+    with no precision indicator. It's suggested that only numbers that fit in
+    64-bit IEEE-754 floating point numbers be stored in here for now, but as the
+    database spec gets revised this may be further split into integers and
+    non-integers.
+
+- `boolean`: This is either true or false.
+
+- `string`: UTF-8
+
+- `bytes`: Binary blobs.
+
+## Type Constructors