API registry and declaration source of truth
API registry and declaration source of truth
- Status: Accepted by ADR 0006. The v1 JSON Schema, Python validator, validator unit tests, and documentation CI check are implemented. The registry currently contains one unsupported Atari ST TIMER research declaration and a 36-entry Amiga GFA BASIC 3.5-specific catalogue: 33 addendum-scoped declarations and three declarations evidenced only by explicitly 3.5-labelled German book Chapter 18. The Amiga entries are book-documented, not implemented as complete runtime features, and have unknown Linux support. Runtime handler resolution is implemented and CI-validated under #115; only FACT, VARIAT, and COMBIN have partial typed registrations, without a claim of Linux support or interpreter conformance. The OKF v0.2 documentation bundle and static Markdown-to-HTML generator are implemented separately; they do not replace the API registry's structured schema or generate API declarations from prose.
- Tracking: issue #12, metadata validation #55 (completed).
1. Purpose and ownership
The checked-in declarative registry is the canonical source for declaration metadata used by parsing/semantic analysis, API profile selection, runtime mapping, reference documentation, and conformance tests.
A registry entry is evidence-backed metadata. Its presence does not mean that the feature is implemented, safe, or supported.
- The registry owns stable declaration identity, spelling, syntax, type/signature metadata, dialect/version/profile availability, evidence, support state, runtime handler IDs, test IDs, and documentation links.
- Rust code implements behavior behind stable registry IDs. It must not independently redefine spelling, syntax availability, signatures, or profile membership.
- Tests remain independent fixtures linked from the registry. Expected results must not be generated by the implementation being tested.
- Generated reference pages may be produced from registry metadata plus authored explanatory material. Generated files must be reproducible and must not be manually edited.
- Research-only, disputed, unknown, unsupported, and unsafe-disabled entries are allowed when their state is explicit.
2. Canonical format and layout
Use TOML as the canonical source format. Use one file per logical module/API group, with a versioned schema.
api/
├── schema/
│ └── registry-v1.json
└── registry/
├── language-core/
│ └── <module>.toml
├── amiga/
│ └── <module>.toml
└── atari-st/
└── <module>.toml
Each file declares schema-version = 1 and contains one or more declaration records. JSON Schema validates the parsed TOML data model; TOML remains the human-edited canonical source. Schema versions are independent of compiler product versions and runtime ABI versions.
Do not put Amiga or Atari declarations in the language-core namespace. Do not create a core API profile: the language core is not a selectable platform API profile.
3. Stable identity and availability
A declaration ID is globally unique, stable across spelling corrections, and independent of version. Use lowercase dot-separated identifiers:
- gfa.language.<module>.<name> for language-core declarations.
- gfa.amiga.<module>.<name> for Amiga API declarations.
- gfa.atari-st.<module>.<name> for Atari ST API declarations.
The declaration record must identify exact source dialect, version availability, and required API profile. Version ranges must use the normalized syntax accepted by the registry schema. Do not use broad labels such as 2.x unless the source evidence explicitly supports the whole range.
A platform API declaration is available only when its required profile is selected. The output target does not enable or disable a source API profile.
4. Required declaration model
| Field | Required | Purpose |
|---|---|---|
| id | Yes | Stable globally unique identity |
| name | Yes | Canonical source spelling |
| kind | Yes | Statement, function, procedure, operator, special form, constant, variable, or structure |
| syntax | Yes | Structured grammar alternatives and contexts |
| availability | Yes | Source dialect, exact versions, and required profile |
| signature | Yes | Ordered parameters, optionality, passing mode, type, and return type |
| semantics | Yes | Contract, side effects, errors, resource lifecycle, and edge cases |
| evidence | Yes | Source ID, edition, page/section, evidence state, and notes |
| implementation | Yes | Implementation state, stable handler ID when applicable, and scope notes that explain partial prototypes or limitations |
| support | Yes | Per-target support and conformance state |
| tests | Yes | Test IDs or explicit pending reason |
| docs | Yes | Canonical reference path and section anchors |
Do not reduce syntax to a display string. The model must express statement/expression context, overloads, optional/repeated arguments, argument passing modes, type suffixes, and special grammar that is not an ordinary function call.
The type vocabulary must distinguish documented source-language types, arrays, pointers/addresses, procedures/functions, target-specific records, and unresolved/opaque values. Never substitute host Rust types for historical GFA BASIC storage semantics without evidence.
Documentation references are repository-local. docs.path must be a canonical repository-relative POSIX path to a regular file under the repository root. Absolute paths, dot/dot-dot or empty components, backslashes, and symlink components are rejected; registry validation must never resolve documentation anchors from arbitrary host files.
5. Evidence, implementation, and support are separate
Do not use one overloaded status field. Each record has independent dimensions.
Evidence status describes the source basis: unknown, inferred, disputed, book-documented, primary-reference-cross-checked, or runtime-observed.
Implementation status describes code: not-implemented, partial, implemented, or unsafe-disabled.
Support classification describes the declared target behavior: supported, partial, adapter-backed, emulated, unsupported, unsafe-disabled, or unknown.
Conformance status describes tests: not-tested, passing, failing, or not-applicable.
A well-documented hardware operation may remain unsafe-disabled. A runtime implementation does not establish historical language conformance. A passing unit test does not establish source-language support unless it is linked to a documented rule or trusted reference behavior.
6. Illustrative declaration
This TOML example illustrates the accepted data model only. It is not an accepted language declaration or an implementation claim.
schema-version = 1
[[declarations]]
id = "gfa.atari-st.time.timer"
name = "TIMER"
kind = "function"
[[declarations.availability]]
dialect = "atari-st-gfa-basic"
versions = ["3.0"]
profile = "atari-st"
[[declarations.syntax]]
form = "call"
contexts = ["expression"]
arguments = []
[declarations.signature]
returns = "integer"
[declarations.semantics]
summary = "Returns elapsed system ticks."
side-effects = []
errors = []
[declarations.evidence]
status = "book-documented"
sources = ["ATARI-GFA3-UM-1988"]
printed-pages = ["chapter 12, pp. 393-452"]
notes = ["Verify OCR-sensitive details against the page image before accepting the exact contract."]
[declarations.implementation]
status = "not-implemented"
[[declarations.support]]
target = "x86_64-unknown-linux-gnu"
classification = "unsupported"
conformance = "not-tested"
tests = []
[declarations.docs]
path = "docs/reference/atari-gfa-basic-3.0.md"
anchors = []
7. Version and profile matching
- Dialect identity and API profile are separate.
- Version availability is explicit and evidence-backed.
- A declaration requiring atari-st is unavailable when that profile is not selected, even if the output target is Linux.
- Profile dependencies must be explicit and source-backed.
- Conflicting visible names across enabled profiles must produce deterministic diagnostics; never resolve silently by file order.
- Version-specific overloads remain separate syntax alternatives or separate declarations linked by a shared concept ID.
- Amiga and Atari ST records remain separate even where historical ports share spelling or implementation lineage.
8. Validation rules
The implemented validator in scripts/validate_api_registry.py and its unit tests currently check:
- JSON Schema structure, required fields, value types, and enumerated status values.
- Namespace prefixes and the required Amiga/Atari profile assignment.
- Duplicate stable declaration IDs.
- Conflicting declaration names for the same dialect/version unless declarations share an explicit overload group.
- Source IDs against the source register.
- Documentation paths and heading anchors.
- Test stable fixture IDs against the sidecar metadata of manifest-listed conformance-corpus entries.
- The presence of a handler ID on implemented or partial declarations.
- Primary/observed evidence, an implementation, passing conformance tests, and documentation anchors before a declaration can claim supported/partial/adapter-backed/emulated support.
- A handler ID for partial, adapter-backed, or emulated support.
The validator does not yet resolve handler IDs to Rust symbols or runtime registrations. It also does not generate reference pages or compare generated output with checked-in pages. The schema and validator do not yet enforce all semantic constraints described by the architecture, including argument ordering rules, full version-range semantics, complete dialect/profile identifier registries, or syntax collision analysis across all enabled profile combinations. Treat those as open work, not implemented guarantees.
The current registry has one research-status Atari ST TIMER declaration and a 36-entry Amiga GFA BASIC 3.5-specific catalogue (33 addendum-scoped declarations plus three declarations evidenced only by explicitly 3.5-labelled German book Chapter 18). The Amiga records are explicitly documented but have no complete implementation or registered executable conformance tests; their Linux support remains unknown. The combinatorics prototype is restricted to literal expressions and is recorded in implementation notes, not promoted to complete support. Existing matrix corpus entries are research-only. MAT INV remains explicitly disputed because the command summary and detailed example reverse the operand direction. Language constructs are kept under language-core/; only Amiga-specific display, window, and GURU operations remain in the Amiga profile. No declaration currently claims support.
9. Documentation generation
The registry is canonical for structured facts. Reference pages may combine generated tables/signatures with authored explanatory text, source citations, and examples.
- Generated sections must be clearly marked and reproducible.
- Hand-authored semantic explanations remain reviewable source material; they must not be overwritten by a generator.
- A generated page must retain the registry ID and source provenance.
- CI must detect stale generated output once the generator exists.
- Copyrighted manual text, scans, and substantial extracts must never be copied into generated documentation.
10. Ownership and workflow
- Research records a candidate and source pages in the relevant reference.
- Add a registry record with evidence status such as inferred, disputed, or book-documented.
- Review spelling, syntax, types, version/profile availability, and semantics.
- Add an independent conformance fixture or record why testing is deferred.
- Link an implementation only after its handler exists.
- Validate schema, conflicts, evidence, docs, handler mappings, and tests.
- Mark evidence as cross-checked only after primary-source review; mark support as supported only after compatible behavior and conformance tests pass.
- Review registry changes with the corresponding language/API documentation and issue.
Acceptance of a registry record does not mean implementation is complete. The registry schema itself is versioned and changes require migration notes.
11. Implementation boundary
ADR 0006 accepts TOML as the canonical registry format, the directory structure above, the stable ID convention, and the separated evidence/implementation/support/conformance model. The v1 JSON Schema, Python validator, validator tests, and CI invocation are implemented. The schema allows implementation notes so partial prototypes and known limitations can be represented without overstating support. The registry contains one unsupported Atari ST TIMER declaration, 32 language-core declarations (29 addendum-scoped and three evidenced only by explicitly 3.5-labelled German book Chapter 18), and four Amiga 3.5 platform declarations; no declaration currently claims supported target behavior, and no runtime handler mapping or documentation generator is implemented. The presence of a research record is not a compiler or runtime feature claim.
Tracking: issue #12, metadata validation #55 (completed), runtime handler resolution #115, conformance corpus #42.
Evidence and status invariants
The validator requires printed-page locators for book-documented and primary-reference-cross-checked declarations, explicit reasons for unknown/unsupported target support and non-passing conformance, and conflict notes for disputed evidence. Disputed evidence cannot claim an implemented behavior, supported target behavior, or passing conformance. Inferred/unknown evidence cannot claim supported target behavior. These checks enforce consistency of recorded claims; they do not prove that a cited page was interpreted correctly.
The public API progress view is generated from the canonical registry by scripts/generate_api_catalogue.py. Its JSON is generated during production and PR-preview builds, not maintained by hand. The view exposes independent evidence, implementation, target-support, and conformance counts and filters declarations by domain/status; it deliberately avoids a misleading aggregate implementation percentage.
Version-scoped evidence
The evidence record includes an optional source-scope field. Amiga declarations claiming version 3.5 must identify their actual evidence scope: amiga-3.5-addendum, amiga-3.5-book-chapter, or runtime-observation; base-manual-3.0 alone cannot justify 3.5 availability. This prevents a declaration from being promoted solely because Chapters 2–12 occur in the same PDF as the 3.5 addendum. A 3.0 baseline inventory is maintained separately until 3.5 applicability is confirmed.
The website payload is split into a compact static index (site/data/api-catalogue.json) and one static JSON detail per declaration (site/data/api-catalogue-details/<id>.json). The browser renders only category and declaration names from the index; full detail DOM and its static file request are deferred until a declaration is expanded. Both payload layers are generated from TOML at build time and checked for deterministic output in CI. Runtime operation requires only static hosting.