Skip to content

feat(stdlib): Sqlite schema introspection + bulk I/O + error inspection — db-theory #1c (6 externs) - #525

Merged
hyperpolymath merged 1 commit into
mainfrom
feat/db-1c-sqlite-introspect-bulk-error
Jun 2, 2026
Merged

feat(stdlib): Sqlite schema introspection + bulk I/O + error inspection — db-theory #1c (6 externs)#525
hyperpolymath merged 1 commit into
mainfrom
feat/db-1c-sqlite-introspect-bulk-error

Conversation

@hyperpolymath

Copy link
Copy Markdown
Owner

Summary

Closes out the SQLite stdlib triad (#522 foundation + #524 prepared statements + this PR). Adds the three remaining practical surfaces:

layer externs
Schema introspection db_schema_tables(d), db_schema_columns(d, table), db_table_exists(d, table)
Bulk CSV I/O db_import_csv(d, table, path, has_header), db_export_csv(d, sql, params, path)
Error inspection db_last_error(d) (typed DbError carrier lands in #1d)

Codegen + adapter

6 b "name" registrations + 6 __as_db* JS helpers, each delegating to one new method on globalThis.__as_sqlite (schemaTables, schemaColumns, tableExists, importCsv, exportCsv, lastError). Real adapters back these with one-line wrappers (PRAGMA table_info, prepare().iterate(), fs writes).

Smoke harness (tests/codegen-deno/sqlite_introspect_bulk.{affine,harness.mjs})

8 smoke functions × 9 assertions — schema_tables ordering, column descriptor shape (incl. PK flag), table_exists true/false, CSV import skipping header, CSV export including header, last_error empty/fault-injected. Mock extends the #1a/#1b adapter with the 6 new methods plus a virtual filesystem map for hermetic CSV testing.

Verification

  • dune build: clean
  • dune runtest: 363 / 363 green
  • tools/run_codegen_deno_tests.sh: 33 / 33 harnesses pass (was 32)

Stacked on #524

#524 (db-theory #1b — prepared statements) MERGED 13:15Z. This branch forks from its head and rebases cleanly.

db-theory #1d (next PR, separate)

  • extern type DbError opaque carrier
  • Result<T, DbError>-shaped variants of the convenience surface
  • Design-blocked on the wider error-model question (do we add Result returns to all extern fns or keep the JSON-string sentinel pattern?)

Test plan

  • dune build clean
  • dune runtest 363/363 green
  • 33/33 codegen-deno harnesses pass
  • 9 introspection/bulk/error smoke assertions pass

🤖 Generated with Claude Code

@hyperpolymath
hyperpolymath enabled auto-merge (squash) June 1, 2026 13:16
…on — db-theory #1c (6 externs)

Closes out the SQLite stdlib triad (#1a foundation + #1b prepared
statements + this PR). Adds the three remaining practical surfaces:
schema reflection, bulk CSV import/export, and a string-form
last-error inspector for the convenience path.

## New stdlib surface (`stdlib/Sqlite.affine`)

### Schema introspection (3 externs)
- `db_schema_tables(d) -> String` — JSON array of table names
  (excludes `sqlite_*` internal namespace).
- `db_schema_columns(d, table) -> String` — JSON array of column
  descriptors `{name, type, notnull, pk}`; `"[]"` if table absent.
- `db_table_exists(d, table) -> Bool` — convenience boolean over the
  tables list.

### Bulk I/O (2 externs)
- `db_import_csv(d, table, csv_path, has_header) -> Int` — assumes the
  destination table exists; returns rows inserted.
- `db_export_csv(d, sql, params_json, csv_path) -> Int` — runs `sql`
  with `params_json`, writes one row per result-row plus a header row;
  returns rows written.

### Error inspection (1 extern)
- `db_last_error(d) -> String` — `""` if no error has been recorded on
  this connection. Disambiguates sentinels (e.g. `db_query_one`
  returning `"null"`) from actual failures. A typed `DbError` carrier
  + `Result<T, DbError>`-shaped variants land in db-theory #1d
  (separate, design-blocked on the wider error-model question).

## Codegen lowerings (`lib/codegen_deno.ml`)

6 new `b "name"` registrations + 6 corresponding `__as_db*` JS
helpers. Each helper is one delegated call to a new method on the
`globalThis.__as_sqlite` adapter (`schemaTables`, `schemaColumns`,
`tableExists`, `importCsv`, `exportCsv`, `lastError`). Real adapters
back these with one-line wrappers: `PRAGMA table_info` for schema,
`Database.prepare().iterate()` + a CSV serialiser for export, etc.

## Smoke harness (`tests/codegen-deno/sqlite_introspect_bulk.{affine,harness.mjs}`)

8 smoke functions × 9 assertions cover:
- `db_schema_tables` returns the two created tables in lexicographic
  order, with `sqlite_*`-prefixed names excluded.
- `db_schema_columns` returns 3 column descriptors; first column
  flagged as PK (mock heuristic mirrors sqlite3's `INTEGER PRIMARY KEY`).
- `db_table_exists` true/false branches.
- `db_import_csv` skips the header row and inserts 3 data rows from
  an in-memory CSV.
- `db_export_csv` writes header + data rows to a virtual filesystem
  path; assertions inspect the written content.
- `db_last_error` returns `""` on a clean connection and the
  fault-injected message after a `RAISE 'simulated failure'` execute.

Mock extends the #1a/#1b in-memory adapter with the 6 new methods
plus a virtual filesystem map (path → string contents) so the CSV
paths exercise no real disk I/O — same isolation discipline as the
existing harnesses.

## Verification

- `dune build bin/main.exe`: clean
- `dune runtest`: **363 / 363** green (codegen ⊆ stdlib consistency
  picks up the 6 new builtins; all match `stdlib/Sqlite.affine`)
- `tools/run_codegen_deno_tests.sh`: **all 33** harnesses pass (was 32)

## Stacked on #524 (db-theory #1b)

#524 finishes the prepared-statement surface; this branch forks from
its head. Both rebase cleanly onto main as they merge.

## db-theory #1d scope (next PR, separate)

- `extern type DbError` opaque carrier + accessor externs.
- `Result<T, DbError>`-shaped variants of `db_open`, `db_execute`,
  `db_query`, `db_query_one`, `db_query_int` (the convenience surface
  where the JSON-string error path is too lossy).
- Design note: prepared-statement surface already raises through the
  AffineScript-side `try`/`catch` desugar, so its `Result`-isation is
  not in scope unless concrete consumers ask.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@hyperpolymath
hyperpolymath force-pushed the feat/db-1c-sqlite-introspect-bulk-error branch from 393c344 to 42b5b54 Compare June 1, 2026 13:31
@github-actions

github-actions Bot commented Jun 1, 2026

Copy link
Copy Markdown

🔍 Hypatia Security Scan

Findings: 88 issues detected

Severity Count
🔴 Critical 2
🟠 High 16
🟡 Medium 70

⚠️ Action Required: Critical security issues found!

View findings
[
  {
    "reason": "Action perpolymath/standards/.github/workflows/governance-reusable.yml@main\n needs attention",
    "type": "unpinned_action",
    "file": "governance.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Action ons/checkout@v6\n    needs attention",
    "type": "unpinned_action",
    "file": "publish-jsr.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Action land/setup-deno@v2\n    needs attention",
    "type": "unpinned_action",
    "file": "publish-jsr.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in affine-vscode-publish.yml",
    "type": "missing_timeout_minutes",
    "file": "affine-vscode-publish.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in casket-pages.yml",
    "type": "missing_timeout_minutes",
    "file": "casket-pages.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in casket-pages.yml",
    "type": "missing_timeout_minutes",
    "file": "casket-pages.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in casket-pages.yml",
    "type": "missing_timeout_minutes",
    "file": "casket-pages.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in ci.yml",
    "type": "missing_timeout_minutes",
    "file": "ci.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in ci.yml",
    "type": "missing_timeout_minutes",
    "file": "ci.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Issue in ci.yml",
    "type": "missing_timeout_minutes",
    "file": "ci.yml",
    "action": "flag",
    "rule_module": "workflow_audit",
    "severity": "medium"
  }
]

Powered by Hypatia Neurosymbolic CI/CD Intelligence

@hyperpolymath
hyperpolymath merged commit d7cdec5 into main Jun 2, 2026
27 checks passed
@hyperpolymath
hyperpolymath deleted the feat/db-1c-sqlite-introspect-bulk-error branch June 2, 2026 09:16
hyperpolymath added a commit that referenced this pull request Jun 2, 2026
Appends today's three db-theory stdlib PRs to the [Unreleased] section.
Repo's changelog-reusable.yml automation isn't wired yet (manual append
per the file's own header pointer).

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
hyperpolymath added a commit that referenced this pull request Jun 2, 2026
## Summary

Refresh the authoritative status surfaces in this repo to reflect the
stdlib **db-theory triplet** that landed yesterday and today:

- PR #525 — `Sqlite.affine` schema introspection + bulk I/O + error
inspection (db-theory #1c, 6 externs)
- PR #526 — `Transaction.affine` (db-theory #2, affine-bounded write-set
isolation, 8 externs)
- PR #527 — `Aggregate.affine` (db-theory #3, SQL group-by + aggregation
primitives, 7 externs)
- PR #528 — CHANGELOG sync (already merged)

`CHANGELOG.md` is already up to date via #528, so this PR touches only
the four files that ledger that work as status, not as history.

## What changed

- `docs/CAPABILITY-MATRIX.adoc` — replace the stale "19/19 stdlib files"
sentence (the AOT gate now discovers files dynamically) and call out the
db-theory triplet as the current authoring frontier alongside Http /
Json / Dict-Map.
- `docs/TECH-DEBT.adoc` §C STDLIB — add a `STDLIB-05` row capturing the
full Sqlite arc (#522 / #524 / #525), Transaction (#526), and Aggregate
(#527), with pointers to the two academic proofs the PRs already shipped
(`docs/academic/proofs/db-theory-{2,3}-*.md`).
- `docs/stdlib-roadmap.adoc` — promote `Sqlite.affine` from `partial /
Coverage audit needed` to `usable`, and add `Aggregate.affine` +
`Transaction.affine` rows to the inventory snapshot.
- `.machine_readable/6a2/STATE.a2ml` — bump `last-updated` to
`2026-06-02` (this file mirrors, it does not lead — per its own
drift-flag).

## What was deliberately *not* touched

- `docs/ROADMAP.adoc` — language/compiler progress; the db-theory PRs
are stdlib-side and don't move the language roadmap.
- A `TEST-NEEDS` file — none exists at repo root; the three PRs each
shipped their own Deno-ESM smoke harness under `tests/codegen-deno/`
(`sqlite_introspect_bulk` / `transaction_smoke` / `aggregate_smoke`) and
the AOT gate auto-picked the new modules up.
- License / SPDX headers — per the estate's no-automated-licence-edits
rule.

## Test plan

- [x] `bash tools/check-doc-truthing.sh` passes locally — banner /
primacy / mirror invariants intact, over-claim ratchet unchanged (no new
baseline entry needed).
- [x] Commit GPG-signed; verified `gpg: Good signature`.

Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant