axon-db

axon-db

🗄️ Database driver library for Axon — PostgreSQL and SQLite drivers, connection pooling, query builder, and migration runner.

Current State: Scaffolding

No implementation code exists yet. This repo is in scaffolding state — the architecture is designed but the first line of implementation code has not been written. The milestones below represent the work to get from zero to a production-quality database library.

The CI pipeline runs but produces no build artifacts yet. Once M15.1 (SQLite FFI) is implemented, the pipeline will run tests against the driver.

Architecture Overview

axon-db provides a typed, idiomatic Axon interface to SQL databases. It is structured in layers:

FFI driver (libsqlite3, libpq)
        ↓
Connection abstraction  (M15.2)
        ↓
Query builder  (M15.3)
        ↓
ORM layer  (M15.7)

Why a layered architecture?

An FFI-only approach (calling C functions directly from Axon) works for trivial cases but breaks down for real applications:

• Each database has a different API (SQLite uses sqlite3_*, PostgreSQL uses PQ*). A connection abstraction unifies them.
• Raw SQL strings are error-prone and non-composable. A query builder provides type-safe, composable query construction.
• Business logic should not construct SQL by hand. An ORM layer maps Axon structs to database rows.

This layered approach means M15.2 (connection abstraction) can be implemented once and reused by both SQLite (M15.1) and PostgreSQL (M15.4) drivers. Similarly, the ORM (M15.7) works over the query builder regardless of which database is underneath.

Milestone Status

Milestone	Feature	Status	Notes
M15.1	SQLite FFI bindings	Not started	First code to land; start here
M15.2	Connection abstraction	Blocked on M15.1	Uniform API over SQLite/PostgreSQL
M15.3	Query builder	Blocked on M15.2	Composable SQL construction
M15.4	PostgreSQL FFI bindings	Blocked on M15.2
M15.5	Connection pooling	Blocked on M15.4	PostgreSQL-only; SQLite is single-connection
M15.6	Migration runner	Blocked on M15.3	Schema versioning and up/down migrations
M15.7	ORM layer	Blocked on M15.3	Struct-to-row mapping

Dependency order: M15.1 → M15.2 → M15.3 → M15.4 → M15.5 → M15.6 → M15.7

M15.3 and M15.4 can proceed in parallel after M15.2 lands.

Milestone Descriptions

M15.1 — SQLite FFI bindings (Critical)

What gets built: Axon extern declarations for sqlite3_open, sqlite3_exec, sqlite3_prepare_v2, sqlite3_step, sqlite3_finalize, sqlite3_close, and the result-row accessors (sqlite3_column_*). Linked against libsqlite3.

API at milestone end: A sqlite_open : (string -> connection) function and basic exec : (connection -> string -> ()) and query : (connection -> string -> rows) operations. No query builder yet.

Minimum viable version: sqlite_open, sqlite_close, sqlite_exec (callback-based), and sqlite3_column_text / sqlite3_column_int64 for reading result rows.

What to read before starting: The axon-lang FFI conventions and extern declaration syntax. Look at how axon-std uses FFI for existing C interop examples.

M15.2 — Connection abstraction (Critical)

What gets built: A Connection interface with open, exec, query, close, and transaction methods. Both SQLite and PostgreSQL drivers implement this interface.

API at milestone end: db_open : (string -> connection) where the URL scheme determines the driver (sqlite://app.db, postgres://user:pass@host/db). db_exec : (connection -> string -> ()). db_query : (connection -> string -> rows). db_close : (connection -> ()).

Why M15.1 must come first: The connection abstraction wraps the raw FFI calls. Without the FFI layer, there is nothing to abstract.

M15.3 — Query builder (High)

What gets built: A chainable query builder DSL. (query_select "users") returns a query object. Methods like where, order_by, limit, offset, join modify it. query_exec runs it against a connection.

API at milestone end:

(let (q (-> (query_select "users")
            (query_where "age > ?" (i64 18))
            (query_order_by "name")
            (query_limit 10)))
  (query_exec q db))

Key design: The query builder must produce parameterized SQL (no string interpolation). ? placeholders map to Axon values passed separately, preventing SQL injection.

SQLite vs PostgreSQL dialects: Both use similar SQL but with minor differences (e.g., RETURNING vs. last insert id). The builder should produce a SQL string + parameter list, and the connection layer handles dialect differences.

M15.4 — PostgreSQL FFI bindings (Medium)

What gets built: Axon extern declarations for libpq functions (PQconnectdb, PQexec, PQgetResult, PQfinish, etc.). Linked against libpq.

Minimum viable version: PQconnectdb, PQexec (for SELECT), and PQntuples / PQgetvalue for reading rows.

Parallel with M15.3: After M15.2, the PostgreSQL driver and query builder can be implemented independently.

M15.5 — Connection pooling (Medium)

What gets built: A Pool type with acquire and release. Pools are per-database URL. PostgreSQL connections are expensive to establish (TCP + auth handshake), so a pool of reusable connections is essential for performance. SQLite is single-threaded and does not benefit from pooling.

Pool interface:

(let (pool (pool_open "postgres://user:pass@host/db" 10))  ; max 10 connections
  (pool_with pool (fn (conn)
    (db_query conn "SELECT * FROM users")))
  (pool_close pool))

Checkout timeout: If all connections are in use, acquire waits up to a configurable timeout (default 30s) then returns an error. See docs/design-decisions.md for the full ADR.

M15.6 — Migration runner (Medium)

What gets built: A migrate module with migrate_add, migrate_run, and migrate_rollback. Migrations are stored in a schema_migrations table keyed by version number. Each migration has an “up” SQL (apply) and “down” SQL (revert).

API at milestone end:

(migrate_add "001_create_users"
  "CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT NOT NULL)"
  "DROP TABLE users")
(migrate_run db)

M15.7 — ORM layer (Low)

What gets built: A deftable macro that declares an Axon struct as a database table, and orm_find, orm_insert, orm_update, orm_delete functions that operate on instances.

API at milestone end:

(deftable User {:id i64 :name string :age i64})

;; Find users older than 18
(let (results (orm_find User {:where {:age (> _ 18)}}))
  ...)

;; Insert a new user
(orm_insert User {:name "Alice" :age 30})

The ORM is built on top of the query builder, so it benefits from everything M15.3 provides.

Planned Architecture

axon-db/
├── src/                         # Axon source files (the library itself)
│   ├── connection.axs           # Connection abstraction (M15.2) — STUB
│   ├── query.axs                # Query builder (M15.3) — STUB
│   ├── pool.axs                 # Connection pool (M15.5) — STUB
│   ├── db.axs                   # Core database abstraction (Connection, Result, Row) — planned
│   ├── sqlite/
│   │   ├── driver.axs           # SQLite FFI bindings (M15.1) — planned
│   │   └── pool.axs             # SQLite connection pool — planned
│   ├── pg/
│   │   ├── driver.axs           # PostgreSQL (libpq) FFI bindings (M15.4) — planned
│   │   └── pool.axs             # Postgres connection pool — planned
│   ├── introspect.axs           # Schema introspection (tables, columns, FKs, indexes) — STUB
│   ├── diagram.axs              # ER diagram generation (Mermaid, DBML) — STUB
│   └── migrate.axs              # Migration runner and versioning (M15.6) — planned
├── tests/                       # Tests
│   ├── test_connection.axs      # Connection tests — STUB
│   ├── test_query.axs           # Query builder tests — STUB
│   ├── test_sqlite.axs          # SQLite driver tests — planned
│   └── test_migrate.axs         # Migration tests — planned
├── docs/
│   ├── architecture.md          # Layer design and interfaces
│   ├── design-decisions.md     # ADRs for pool, transactions, etc.
│   ├── testing-strategy.md     # SQL fixtures, mock drivers
│   ├── api/                     # Planned API docs (Axon formatter strips comments)
│   │   ├── README.md
│   │   ├── connection.md
│   │   ├── query.md
│   │   └── pool.md
│   ├── introspect.md            # Schema introspection API reference
│   ├── diagram.md               # Diagram generation reference
│   └── ...                      # orm.md, migrations.md, etc.
├── axon_db/                     # Python reference implementation
│   ├── __init__.py
│   └── orm.py                   # ORM layer (M15.7)
├── migrations/                  # SQL migration files
├── tools/                       # Build/CI helper scripts
│   ├── introspect.py            # Schema introspection CLI tool
│   ├── test_introspect.py       # Introspection unit tests
│   └── test_introspect_properties.py  # Property-based tests
├── CONTRIBUTING.md              # Contributor guide (start here!)
├── .gitlab-ci.yml
└── README.md

Module Status

Module	Status	Milestone	API Doc
src/connection.axs	STUB	M15.2	docs/api/connection.md
src/query.axs	STUB	M15.3	docs/api/query.md
src/pool.axs	STUB	M15.5	docs/api/pool.md
src/introspect.axs	STUB	—	docs/introspect.md
src/diagram.axs	STUB	—	docs/diagram.md
src/redis/	Python impl	#29	docs/api/redis.md
src/s3/	Python impl (SigV4 core)	#32	docs/api/s3.md

The stub files are minimal compilable Axon modules (each returns i32 0) with detailed comments documenting the planned API. They activate the previously-dormant test:axon-* CI jobs so the pipeline will run golden/type/format checks as soon as real implementations land.

Python Reference Implementation

The axon_db/ directory contains a Python reference implementation of the ORM layer (M15.7). This serves as:

1. A specification for the Axon implementation (what the API should look like)
2. A testable target — the Python tests verify the API contract
3. A fallback — apps that need a DB now can use the Python version while the Axon version is under construction

The Python and Axon implementations should stay in sync API-wise.

Schema Introspection & Diagram Generation

The tools/introspect.py module provides SQLite schema introspection and ER diagram generation. It serves as the Python prototype for src/introspect.axs and src/diagram.axs.

CLI Usage

# Generate Mermaid ER diagram
python tools/introspect.py mydb.sqlite --format mermaid

# Generate DBML diagram
python tools/introspect.py mydb.sqlite --format dbml

# List all tables
python tools/introspect.py mydb.sqlite --tables

# Describe a table
python tools/introspect.py mydb.sqlite --describe users

# Show foreign keys
python tools/introspect.py mydb.sqlite --fks posts

# Show indexes
python tools/introspect.py mydb.sqlite --indexes posts

See docs/introspect.md and docs/diagram.md for full API reference.

Ecosystem

axon-db is part of the Axon ecosystem:

Package	Description	Status
axon-lang	Compiler & runtime	✅ Active
axon-std	Standard library	✅ Active
axon-debug	GDB/LLDB debug tools	✅ Active
axon-db	Database drivers	📋 Scaffolding

Contributing

See CONTRIBUTING.md for:

• How to implement M15.1 (SQLite FFI) — the first code to land
• Axon FFI primer: writing extern declarations for C functions
• Development environment setup
• Test strategy for the FFI layer

License

See project license file.