# Bio-Digital Architecture Blueprint

Project: Bio-Digital  
Extension: `.bio`  
Engine Role: Native neuromorphic control language for the NRL engine  
Execution Layer: C, x86-64, AArch64, AVX2, AVX-512 where available, NEON  
Non-Goals: Python runtimes, VM bytecode, ASTs, object systems, high-level wrappers

## 0. Architectural Position

Bio-Digital is a low-level neural execution language whose tokens map directly to precompiled C function pointers backed by architecture-specific machine-code kernels. The language is not a general-purpose programming language. It is a deterministic neural control surface for GGUF-backed inference, sparse routing, memory traversal, pruning, and continuous delta learning.

The design target is not literal 1.5 PetaOps on ordinary hardware. The practical target is 1.5 PetaOps equivalent neural throughput by combining INT4 packing, compute avoidance, sparse dispatch, GGUF memory mapping, hierarchical routing, and zero-copy tensor access. Real throughput is bounded by the installed CPU, memory bandwidth, cache topology, SIMD width, and optional accelerator backends added later.

Bio-Digital avoids traditional abstractions:

- No Python.
- No AST.
- No VM.
- No heap object graph for language values.
- No traditional variables.
- No tensor serialization layer.
- No intermediate representation between token recognition and dispatch.

The `.bio` file is a compact command stream. The C lexer scans tokens, resolves each token through a fixed opcode table, and dispatches immediately into a native kernel entrypoint.

## 0.1 NRL Amalgamation Rules

Bio-Digital may amalgamate the valuable native C patterns from the existing NRL engine, but only as a machine-level substrate. NRL's native engine is a reference and kernel source, not a Python runtime to extend.

Allowed native NRL patterns:

- Packed INT4 memory layout and nibble operations.
- ZPM compute-avoidance model with exact executed-versus-baseline accounting.
- Omega hierarchical sparse routing and active-frontier masks.
- Init-time CPU feature dispatch with no branching inside hot loops.
- Stable C ABI discipline using versioned `bio_v1_*` symbols.
- AVX2 and NEON kernel style with scalar reference paths for tests only.
- Native build and test flow modeled after `build.ps1` and `build.sh`.

Forbidden inheritance:

- No `nrlpy` runtime dependency.
- No Python GGUF orchestration.
- No Python chat runner.
- No subprocess-based execution path.
- No manifest-first execution model for `.bio`.
- No high-level wrapper dependency in the Bio-Digital hot path.

The execution chain must remain:

```text
.bio bytes
  -> native C lexer
  -> opcode function pointer
  -> C handler
  -> pre-bound AVX2 / NEON / scalar-test kernel
  -> mapped GGUF INT4 lattice
```

## 1. Part 1: Bio-Digital Master Blueprint

### 1.1 Core Execution Principles

1. Tokens are opcodes.
2. Opcodes are direct indexes into immutable dispatch tables.
3. Operands are immediate scalar fields, file descriptors, offsets, lattice coordinates, or kernel flags.
4. GGUF tensors are memory-mapped and viewed as packed INT4 lattice segments.
5. The model state is addressed as a continuous 16000D manifold, not as named variables.
6. Sparse routing decides what not to compute before hot-loop execution.
7. Learn mode applies bounded delta updates into a sidecar lattice rather than mutating the read-only GGUF mapping.
8. Chat inference is a native opcode path, not an external runner.

### 1.2 Opcode Instruction Set

The first Bio-Digital ISA should remain small and stable. Each opcode has a fixed binary handler signature in C and maps to a precompiled kernel family.

| Opcode | Purpose | Native Handler Class |
| --- | --- | --- |
| `ABSORB_GGUF` | Open and mmap a GGUF file, parse metadata, map tensor spans into the INT4 lattice view. | File mapping, GGUF header parser, tensor span registrar |
| `MANIFOLD_ADDR` | Set the active 16000D coordinate window for subsequent neural operations. | Coordinate decoder, lattice span selector |
| `PULSE` | Execute one neural activation pulse over the active manifold slice. | INT4 dot/dequant hot loop, activation approximation |
| `PRUNE` | Remove inactive or low-value routes from the current execution frontier. | Sparse mask update, threshold compaction |
| `ZPM_SKIP` | Zero-point matrix skip. Avoid computation for blocks whose quantized zero/near-zero structure cannot affect output above threshold. | Block metadata scanner, skip-mask generator |
| `OMEGA_ROUTE` | Hierarchical sparse routing across manifold regions and attention heads/layers. | Router kernel, top-k frontier selector |
| `LEARN_CYCLE` | Start, step, or stop bounded background learning using delta-weight sidecar pages. | Delta update kernel, scheduler hook |
| `INFER_CHAT` | Run low-latency token inference using the current lattice, sparse route state, and chat buffer. | Token loop, sampler, KV cache hot path |
| `DELTA_COMMIT` | Merge approved delta sidecar blocks into a writable learned lattice image. | Delta quantizer, page writer |
| `CACHE_BIND` | Bind KV cache, route cache, and manifold window cache to specific memory arenas. | Arena allocator, cache-line aligned binding |
| `SYNC_FENCE` | Insert an explicit memory and thread synchronization barrier. | Atomic fence, worker rendezvous |
| `HALT` | End execution and release transient mappings. | Cleanup, fd close, arena release |

### 1.3 Opcode Shape

Each textual opcode has a fixed operand schema. The lexer does not build an AST. It validates the token and immediately calls the handler with a compact operand frame.

Example conceptual handler shape:

```c
typedef int (*bio_op_fn)(bio_context_t *ctx, const bio_operands_t *ops);
```

The dispatch table is static:

```c
static const bio_op_fn BIO_DISPATCH[BIO_OPCODE_COUNT] = {
    bio_absorb_gguf,
    bio_manifold_addr,
    bio_pulse,
    bio_prune,
    bio_zpm_skip,
    bio_omega_route,
    bio_learn_cycle,
    bio_infer_chat,
    bio_delta_commit,
    bio_cache_bind,
    bio_sync_fence,
    bio_halt
};
```

The lexer maps token bytes to opcode indexes through a fixed perfect hash or trie. Once matched, control transfers to the handler. There is no parse tree.

### 1.4 Memory Addressing Model: INT4 Packed Lattice

Bio-Digital treats model memory as a continuous 16000D mathematical manifold. This is an addressing discipline, not a claim that all dimensions are physically materialized. Physical memory remains linear. The 16000D coordinate is folded into a linear packed INT4 address.

Logical address:

```text
L = <d0, d1, d2, ... d15999>
```

Physical address:

```text
byte_offset = base + fold16000(L, tensor_span, stride_map)
nibble_sel  = byte_offset & 1
```

INT4 packing:

- Two signed or affine-quantized INT4 values per byte.
- Low nibble stores element `2n`.
- High nibble stores element `2n + 1`.
- Tensor scale, zero point, group size, and quantization type are read from GGUF metadata.
- Dequantization is performed inside SIMD hot loops only when required.
- Sparse masks are stored separately as block bitsets, not interleaved with tensor nibbles.

GGUF mapping model:

```text
GGUF mmap region
  header
  metadata table
  tensor directory
  tensor data spans

Bio lattice view
  tensor_span[0] -> manifold region R0
  tensor_span[1] -> manifold region R1
  tensor_span[n] -> manifold region Rn

Sidecar learn lattice
  delta_span[0] -> learned adjustment for R0
  delta_span[n] -> learned adjustment for Rn
```

The GGUF file should usually remain mapped read-only. Learning uses a sidecar delta lattice:

- Read path: `effective_weight = gguf_int4_weight + delta_int4_or_int8_adjustment`.
- Commit path: optional explicit `DELTA_COMMIT`.
- Crash safety: sidecar pages can be journaled before commit.
- Reproducibility: base GGUF remains unchanged unless explicitly exported.

### 1.5 Lattice Regions

The initial lattice should define fixed region classes:

| Region | Contents |
| --- | --- |
| `R.EMBED` | Token embedding tensors |
| `R.ATTN_Q` | Attention query tensors |
| `R.ATTN_K` | Attention key tensors |
| `R.ATTN_V` | Attention value tensors |
| `R.ATTN_O` | Attention output tensors |
| `R.FFN_UP` | Feed-forward up projection |
| `R.FFN_GATE` | Feed-forward gate projection |
| `R.FFN_DOWN` | Feed-forward down projection |
| `R.NORM` | Normalization tensors |
| `R.KV_CACHE` | Runtime KV cache arena |
| `R.ROUTE` | Sparse route masks and active frontiers |
| `R.DELTA` | Learn-mode sidecar deltas |

`MANIFOLD_ADDR` selects region, layer, head, block, stride, and optional coordinate hints. The coordinate folding function must be deterministic and architecture independent.

### 1.6 Complete `.bio` Example

This example memory-maps a GGUF model, binds native caches, starts background learning, applies sparse routing, and runs chat inference.

```text
# Bio-Digital neural command stream
# No variables. No AST. Tokens dispatch directly to native handlers.

ABSORB_GGUF path="models/nrl-core-q4.gguf" mode=readonly quant=int4 map=direct

CACHE_BIND arena=kv size=512m align=64 numa=local
CACHE_BIND arena=route size=64m align=64 numa=local
CACHE_BIND arena=delta size=256m align=4096 mode=sidecar

MANIFOLD_ADDR region=R.EMBED layer=0 block=0 dims=16000 fold=gguf
ZPM_SKIP threshold=0.0008 block=64 mode=prepass

MANIFOLD_ADDR region=R.ROUTE layer=all head=all block=active dims=16000 fold=omega
OMEGA_ROUTE depth=4 fanout=8 topk=2 entropy_gate=0.11
PRUNE threshold=0.015 keep=frontier

LEARN_CYCLE mode=start cadence=background delta=R.DELTA rate=0.00003 clamp=0.002

MANIFOLD_ADDR region=R.ATTN_Q layer=active head=active block=frontier dims=16000 fold=gguf
PULSE kind=attention sparse=route simd=auto

INFER_CHAT prompt="Explain the NRL engine in one paragraph." max_tokens=128 temperature=0.7 route=omega latency=low

LEARN_CYCLE mode=step source=chat delta=R.DELTA retain=bounded
SYNC_FENCE scope=learn
HALT
```

## 2. Part 2: C-Level Dispatch Architecture

### 2.1 Runtime Components

Bio-Digital should be implemented as a small native C engine:

| Component | Responsibility |
| --- | --- |
| Lexer | Reads `.bio` bytes, identifies opcode tokens, parses fixed operand fields |
| Opcode Table | Maps opcode IDs directly to function pointers |
| GGUF Mapper | Opens, validates, and memory-maps GGUF tensors |
| Lattice Mapper | Converts GGUF tensor spans into manifold regions |
| SIMD Kernel Table | Selects AVX2, AVX-512, or NEON handlers at startup |
| Route Engine | Maintains Omega sparse frontiers and ZPM skip masks |
| Learn Scheduler | Runs bounded background delta updates |
| Chat Runner | Executes token loop, KV cache updates, sampling, and route reuse |
| Arena Manager | Provides aligned memory regions for KV cache, routes, and deltas |

### 2.2 Lexer and Dispatch Flow

The lexer operates as a single forward scan:

```text
read bytes
skip whitespace/comments
match opcode token
parse fixed operands into stack frame
dispatch handler immediately
advance to next token
```

There is no AST allocation. There is no IR lowering pass. There is no bytecode interpreter. The only runtime indirection is the function pointer selected from the opcode dispatch table.

Dispatch path:

```text
.bio bytes
  -> token matcher
  -> opcode id
  -> operand frame
  -> BIO_DISPATCH[opcode_id](ctx, operands)
  -> architecture-specific kernel table
  -> AVX2/AVX-512/NEON hot loop
```

### 2.3 GGUF Absorption

`ABSORB_GGUF` must:

1. Open the GGUF path with native OS file APIs.
2. Validate GGUF magic, version, metadata count, and tensor count.
3. Memory-map the file read-only.
4. Build a compact tensor directory referencing mapped offsets.
5. Register tensor spans into lattice regions.
6. Read quantization metadata, group size, scale, and zero-point information.
7. Refuse unsupported quantization formats unless a native kernel exists.

No serialization occurs. No Python object loads the model. Tensor data stays in the mapped file and is consumed directly by native kernels.

### 2.4 SIMD Hot Loops

Kernel selection happens once at startup:

```text
x86-64:
  AVX-512 VNNI if available
  AVX2 fallback
  scalar debug fallback only for validation

AArch64:
  NEON primary path
  SVE optional future path
```

Hot loops should:

- Decode INT4 nibbles in vector batches.
- Apply quantization scales by group.
- Fuse dequantization, dot product, activation, and route mask application where possible.
- Skip blocks before decode when ZPM masks prove the block inactive.
- Use branchless inner loops and move branching to block-level dispatch.
- Align arenas to cache-line boundaries.
- Prefetch upcoming tensor and KV cache spans.
- Keep sparse frontier masks compact enough for L1/L2 cache residency.

### 2.5 Minimal Branching Strategy

Branching should be moved out of the inner loop:

- Feature detection happens once.
- Opcode dispatch happens once per command.
- Quantization type dispatch happens once per tensor block family.
- Route decisions happen at block granularity.
- Inner loops consume precomputed masks and strides.

The hot path should be shaped like:

```text
for each active block:
  load packed INT4
  load scale/zero metadata
  load route mask
  decode nibbles
  fused multiply/accumulate
  apply activation approximation
  write output or KV cache
```

## 3. Learn Mode and Chat Runner

### 3.1 Learn Mode

`LEARN_CYCLE` simulates child-like growth through bounded continuous adaptation. It must not uncontrolledly mutate the mapped GGUF. Instead, it writes into a sidecar delta lattice.

Learning constraints:

- Delta updates are clamped.
- Updates are sparse.
- Updates are tied to active routes only.
- Background learning must yield to `INFER_CHAT`.
- The scheduler must enforce a strict latency budget.
- Deltas should be versioned or journaled if persistence is enabled.

Learning modes:

| Mode | Behavior |
| --- | --- |
| `start` | Enable background delta scheduler |
| `step` | Run one bounded update pass |
| `pause` | Stop scheduling new updates |
| `commit` | Prepare sidecar blocks for `DELTA_COMMIT` |
| `stop` | End learning and flush scheduler state |

### 3.2 Chat Runner

`INFER_CHAT` owns the token loop:

1. Tokenize prompt through a native tokenizer loaded from GGUF metadata or native tokenizer file.
2. Embed tokens from mapped tensor spans.
3. Apply Omega routing to select active manifold regions.
4. Apply ZPM skip masks.
5. Run attention and FFN pulse kernels.
6. Update KV cache.
7. Sample next token.
8. Repeat until stop condition.

The chat runner should reuse route decisions across nearby tokens when entropy and activation drift remain below thresholds.

## 4. Part 3: Prompt Execution Roadmap

The build should proceed in four strict phases. Each phase must produce a small, reviewable native artifact. Do not start with full inference. Start with the file format, dispatch skeleton, and correctness harness.

### Phase 1: Native Project Skeleton and Opcode Lexer

Goal:

- Create the C project layout inside `Bio-Digital`.
- Define `.bio` opcode names and operand schemas.
- Implement a direct lexer-to-dispatch skeleton.
- Add a scalar validation path only for correctness tests, not performance claims.
- No GGUF tensor execution yet.

Exact Prompt 1:

```text
Build Phase 1 for Bio-Digital. PURE MACHINE CODE FOCUS. ZERO ABSTRACTION.

Create the native C project skeleton inside the Bio-Digital folder only. Do not touch nrlpy. Implement the `.bio` lexer and direct opcode dispatch skeleton with no AST, no VM, no bytecode, and no Python. Define the initial opcode table for ABSORB_GGUF, MANIFOLD_ADDR, PULSE, PRUNE, ZPM_SKIP, OMEGA_ROUTE, LEARN_CYCLE, INFER_CHAT, DELTA_COMMIT, CACHE_BIND, SYNC_FENCE, and HALT. The lexer must scan a `.bio` file, recognize opcode tokens, parse fixed operands into compact C structs, and immediately call the matching function pointer. Stub handlers may log deterministic native events for now. Add a minimal CMake or Make build and one sample `.bio` file that exercises all opcodes.
```

### Phase 2: Native GGUF mmap and INT4 Lattice View

Goal:

- Implement `ABSORB_GGUF`.
- mmap GGUF read-only.
- Parse GGUF header, metadata directory, tensor directory, and tensor spans.
- Register tensor spans into lattice region records.
- Add INT4 packed addressing helpers.

Exact Prompt 2:

```text
Build Phase 2 for Bio-Digital. PURE MACHINE CODE FOCUS. ZERO ABSTRACTION.

Implement native GGUF absorption in C inside the Bio-Digital project. Do not use Python, external high-level model loaders, serialization layers, or wrappers. ABSORB_GGUF must open and mmap a GGUF file read-only, validate the header, parse metadata and tensor directory records, and register tensor data spans into a 16000D folded INT4 lattice view. Add packed INT4 nibble addressing helpers, tensor span structs, quantization metadata records, and strict error handling for unsupported GGUF versions or quantization formats. Keep the lexer-to-function-pointer dispatch design intact and do not introduce an AST, IR, or VM.
```

### Phase 3: Sparse Routing, ZPM Skip, and SIMD Kernel Table

Goal:

- Add architecture feature detection.
- Build kernel function tables for AVX2 and NEON.
- Implement ZPM block skipping and Omega sparse routing masks.
- Add scalar correctness comparison for the hot loops.

Exact Prompt 3:

```text
Build Phase 3 for Bio-Digital. PURE MACHINE CODE FOCUS. ZERO ABSTRACTION.

Add native sparse execution support to Bio-Digital. Implement CPU feature detection and architecture-specific kernel selection for x86-64 AVX2 and AArch64 NEON, with a scalar validation path only for tests. Implement ZPM_SKIP as block-level compute avoidance over packed INT4 tensor spans. Implement OMEGA_ROUTE as a hierarchical sparse frontier selector that produces compact route masks consumed by PULSE. Keep branching outside the innermost loops, preserve direct opcode dispatch, and do not add Python, high-level wrappers, an AST, an IR, or a bytecode VM.
```

### Phase 4: Learn Cycle and Native Chat Runner

Goal:

- Add delta sidecar lattice.
- Implement bounded background learn scheduler.
- Implement native `INFER_CHAT` control loop.
- Integrate KV cache arena, route reuse, and sparse pulse execution.

Exact Prompt 4:

```text
Build Phase 4 for Bio-Digital. PURE MACHINE CODE FOCUS. ZERO ABSTRACTION.

Implement Bio-Digital learn mode and the native chat runner. Add a sidecar delta lattice for LEARN_CYCLE so the read-only GGUF mmap remains immutable unless DELTA_COMMIT is explicitly invoked. Implement bounded sparse delta updates with clamp limits, background scheduling, and synchronization fences. Implement INFER_CHAT as a native C token loop that uses the mapped GGUF lattice, KV cache arena, Omega route masks, ZPM skip masks, and PULSE kernels. Keep the latency path native and direct: no Python, no high-level inference wrapper, no AST, no IR, and no VM. Include focused native tests or sample command streams that prove dispatch, mmap, routing, learning, and chat execution paths are connected.
```

## 5. Build Discipline

Every implementation phase must obey these rules:

- Keep code inside `Bio-Digital`.
- Do not modify `nrlpy`.
- Do not use Python for runtime execution.
- Do not call Python, `nrlpy`, or wrapper libraries from the Bio-Digital hot path.
- Do not use subprocesses as a runtime execution mechanism.
- Do not make `.bio` a manifest-first format; `.bio` remains an opcode stream.
- Do not introduce a VM or AST.
- Use versioned native C symbols under the `bio_v1_*` namespace.
- Reuse NRL native engine patterns only where they remain C-level and machine-near.
- Prefer fixed-size structs and explicit memory ownership.
- Fail closed on malformed `.bio` and unsupported GGUF features.
- Keep debug scalar paths separate from production SIMD paths.
- Measure real throughput separately from equivalent skipped-op throughput.
- Treat the GGUF mmap as read-only unless an explicit export or commit mode is designed.

## 6. Acceptance Criteria

The architecture is acceptable when the project can demonstrate:

1. A `.bio` file can be scanned opcode by opcode.
2. Each opcode dispatches directly to a C function pointer.
3. `ABSORB_GGUF` mmaps a GGUF without Python or serialization.
4. GGUF tensor spans are represented as folded 16000D INT4 lattice regions.
5. Sparse route masks can prevent inactive blocks from entering hot loops.
6. SIMD kernel tables select AVX2 or NEON handlers without runtime branching in inner loops.
7. Learn mode writes bounded delta updates into sidecar memory.
8. `INFER_CHAT` is owned by the native engine, not by `nrlpy`.

## 7. Final Architectural Rule

Bio-Digital must be treated as the native nervous system layer for NRL, not as a Python package and not as a scripting frontend. The `.bio` language is a machine-near command stream whose purpose is to bind GGUF model memory, sparse neural routing, SIMD execution, background learning, and chat inference into one C-level execution path.

## 8. License And IP Rule

Bio-Digital remains under the repository root `LICENSE`: `RomanAILabs Proprietary Source-Available Evaluation License 1.0`.

Do not add MIT, Apache, BSD, GPL, public-domain, or other permissive/open-source license terms to Bio-Digital. Bio-Digital is part of the NRL / RomanAILabs protected work, and all rights are reserved by Daniel Harding / RomanAILabs.

## 9. Production Product Target

The production Bio-Digital product is not a manual smoke runner. It must be an automated native system that can:

1. Fully ingest a GGUF model into a persistent Bio-Digital model state.
2. Build a reusable tensor, tokenizer, route, KV, and delta index.
3. Start an automated long-running learn cycle without user supervision.
4. Load chat instantly from the persistent state while learning continues in the background.
5. Detect 60 seconds of chat idle time and resume background learning automatically.
6. Persist learning into a protected `.bdl` sidecar without mutating the read-only GGUF.
7. Resume from checkpoints after process restart or crash.
8. Produce coherent native chat without `nrlpy`, Python, llama.cpp wrappers, subprocesses, VM bytecode, or AST execution.

The current Phase 4 implementation proves the opcode path, mmap path, sparse routing path, sidecar delta memory, and native chat control loop. It is not the final product. Production requires the phases below.

## 10. Automated Runtime Modes

Bio-Digital must support three production runtime modes.

### 10.1 Full Ingest Mode

Opcode:

```text
INGEST_GGUF path="..." out="D:\model.bdst" sidecar="D:\model.bdl" mode=full
```

Responsibilities:

- mmap the GGUF read-only.
- Parse all GGUF metadata, tokenizer tables, and tensor records.
- Build a tensor-role index for embeddings, attention, FFN, norms, output, and tokenizer assets.
- Build quantization descriptors for Q4_K_M, Q6_K, F16, and F32 tensors.
- Build route metadata for ZPM and Omega.
- Build an on-disk Bio-Digital State Table (`.bdst`) containing offsets, hashes, tensor roles, supported kernels, and checkpoint metadata.
- Initialize a `.bdl` sidecar delta file with zero sparse pages.
- Write a model SHA256 and schema version into every persistent artifact.

Full ingest is expected to be a minutes-to-hours operation depending on model size and hardware. It should be resumable and should never modify the GGUF.

### 10.2 Autonomous Learn Mode

Opcode:

```text
AUTO_LEARN state="D:\model.bdst" sidecar="D:\model.bdl" budget_hours=24 checkpoint_sec=300 idle_only=false
```

Responsibilities:

- Run the native learn scheduler without user prompts.
- Traverse the tensor-role graph in deterministic blocks.
- Apply ZPM skip and Omega sparse routing before compute.
- Produce bounded delta updates into `.bdl`.
- Checkpoint every `checkpoint_sec`.
- Track progress as percent of tensor blocks processed, delta pages touched, route coverage, and estimated time remaining.
- Enforce clamp, rollback, and checksum invariants.
- Never mutate the GGUF.

The old architecture estimated 12-24 hours for a meaningful full learn pass. Production Bio-Digital should expose that estimate explicitly:

```text
learn_eta = baseline_blocks / measured_blocks_per_sec
```

The estimate must be recalculated during learning from real measured throughput, not hard-coded.

### 10.3 Chat With Idle Background Learning

Opcode:

```text
CHAT_RUN state="D:\model.bdst" sidecar="D:\model.bdl" idle_learn_after_sec=60 stream=true
```

Responsibilities:

- Load `.bdst` and `.bdl` immediately.
- Start the chat loop with no full re-ingest.
- Run native tokenizer, transformer forward pass, sampler, KV cache, and sparse route reuse.
- Pause background learning during active token generation.
- If the user is idle for 60 seconds, resume background learn cycles.
- Stop background learning immediately when the next user prompt arrives.
- Checkpoint sidecar deltas before shutdown.

This is the target user experience:

```powershell
.\build\bin\bio.exe ingest "C:\Users\Asus\Desktop\Documents\RomaPy Engine\phi-3-mini-4k-instruct.Q4_K_M.gguf" --out "D:\phi3.bdst" --delta "D:\phi3.bdl"
.\build\bin\bio.exe learn "D:\phi3.bdst" --hours 24
.\build\bin\bio.exe chat "D:\phi3.bdst"
```

No `nrlpy` command is part of the production workflow.

## 11. Production File Formats

### 11.1 `.bdst` Bio-Digital State Table

The `.bdst` file is the fast-load index for a GGUF model.

Required fields:

- Magic: `BDST`.
- Version.
- GGUF path hash.
- GGUF SHA256.
- Model architecture.
- Context length.
- Embedding length.
- Layer count.
- Head count.
- KV head count.
- Feed-forward length.
- Vocabulary size.
- Tokenizer section offset.
- Tensor-role table offset.
- Quantization descriptor table offset.
- Route metadata offset.
- Last ingest checkpoint.

The `.bdst` file does not duplicate full tensor data. It indexes the read-only GGUF and persistent sidecar.

### 11.2 `.bdl` Bio-Digital Delta Lattice

The `.bdl` file stores learned sidecar updates.

Required fields:

- Magic: `BDL`.
- Version.
- GGUF SHA256.
- Delta page size.
- Sparse page count.
- Learn steps.
- Last checkpoint timestamp.
- Checksum.
- Rollback journal offset.
- Sparse delta pages.

Delta pages must be sparse, bounded, checksummed, and crash-safe.

## 12. Production Scheduler

The native scheduler must be C-only and event driven.

States:

| State | Behavior |
| --- | --- |
| `BOOT` | Load `.bdst`, mmap GGUF, map `.bdl`. |
| `INGEST` | Build or resume `.bdst`. |
| `LEARN_ACTIVE` | Run background or dedicated learning. |
| `CHAT_ACTIVE` | Pause learning and serve tokens. |
| `CHAT_IDLE` | Wait for 60 seconds of idle. |
| `IDLE_LEARN` | Resume bounded background learning. |
| `CHECKPOINT` | Flush `.bdl` journal and checksums. |
| `HALT` | Flush, unmap, close handles. |

Idle rule:

```text
if now - last_user_input >= 60 seconds:
    enter IDLE_LEARN
else:
    remain CHAT_ACTIVE or CHAT_IDLE
```

Learning must yield immediately when user input arrives.

## 13. Production Build Roadmap

### Phase 5: Native Tokenizer, Metadata, And Tensor Resolver

Goal:

- Parse tokenizer metadata from GGUF.
- Parse architecture metadata.
- Build tensor-role lookup tables.
- Replace first-tensor selection with explicit layer/role selection.
- Add `bio inspect` to print model structure from native C.

Exact Prompt 5:

```text
Build Phase 5 for Bio-Digital. PURE MACHINE CODE FOCUS. ZERO ABSTRACTION.

Implement native GGUF tokenizer/config extraction and tensor-role resolution in Bio-Digital. Do not use Python, nrlpy, llama.cpp wrappers, subprocesses, an AST, an IR, or a VM. Parse model architecture, context length, embedding size, layer count, attention heads, KV heads, FFN size, vocabulary size, tokenizer tokens, and tensor names from the mapped GGUF. Build a C tensor-role resolver for embeddings, attention, FFN, norms, output, and tokenizer assets. Add native tests and a `bio inspect` path that prints the parsed structure.
```

### Phase 6: Real Quant Kernels

Goal:

- Implement real Q4_K_M and Q6_K block decode.
- Implement scalar reference matmul.
- Implement AVX2 matmul/dequant hot paths.
- Add parity tests against fixed block fixtures.

Exact Prompt 6:

```text
Build Phase 6 for Bio-Digital. PURE MACHINE CODE FOCUS. ZERO ABSTRACTION.

Implement real native Q4_K_M and Q6_K GGUF block decode and matmul kernels. Keep scalar reference kernels for tests and add AVX2 production kernels selected at init time. Do not use Python, nrlpy, llama.cpp wrappers, subprocesses, an AST, an IR, or a VM. Add fixed binary fixtures for Q4_K_M and Q6_K blocks, parity tests, and executed-versus-baseline accounting.
```

### Phase 7: Native Transformer Forward Pass

Goal:

- Implement Phi-3 compatible forward pass.
- Implement embedding lookup, RMSNorm, RoPE, attention, KV cache, FFN gate/up/down, residuals, and logits.
- Keep all token-loop buffers preallocated.

Exact Prompt 7:

```text
Build Phase 7 for Bio-Digital. PURE MACHINE CODE FOCUS. ZERO ABSTRACTION.

Implement the native Phi-3 transformer forward pass inside Bio-Digital. Use the mapped GGUF tensors and native Q4_K_M/Q6_K kernels only. Implement embedding lookup, RMSNorm, RoPE, attention, KV cache, FFN gate/up/down, residual connections, and logits. No Python, no nrlpy, no llama.cpp wrappers, no subprocesses, no AST, no IR, and no VM. Add native correctness tests for tensor shape validation, KV cache behavior, and one-token forward execution.
```

### Phase 8: Native Sampler And Real Chat

Goal:

- Replace smoke chat output with real sampled tokens.
- Implement temperature, top-k, top-p, repeat penalty, stop tokens, and streaming.
- Add `bio chat`.

Exact Prompt 8:

```text
Build Phase 8 for Bio-Digital. PURE MACHINE CODE FOCUS. ZERO ABSTRACTION.

Replace the current Bio-Digital smoke chat path with real native sampled text generation. Implement temperature, top-k, top-p, repeat penalty, stop-token handling, seed control, token streaming, and a `bio chat` command. Use only the native transformer forward pass and mapped GGUF tensors. No Python, no nrlpy, no llama.cpp wrappers, no subprocess runtime, no AST, no IR, and no VM.
```

### Phase 9: Full Ingest And Persistent State

Goal:

- Add `.bdst` state table.
- Add resumable full ingest.
- Add `.bdl` sparse delta persistence with rollback journal.
- Add model SHA256 binding.

Exact Prompt 9:

```text
Build Phase 9 for Bio-Digital. PURE MACHINE CODE FOCUS. ZERO ABSTRACTION.

Implement full automated GGUF ingest and persistent Bio-Digital state. Add `.bdst` state table generation, GGUF SHA256 binding, tensor-role indexes, tokenizer indexes, quantization descriptor tables, route metadata, resumable ingest checkpoints, and `.bdl` sparse delta persistence with checksums and rollback journal. The base GGUF must remain read-only. Do not use Python, nrlpy, wrappers, subprocesses, an AST, an IR, or a VM.
```

### Phase 10: Autonomous Learn Scheduler

Goal:

- Add automated 12-24 hour learn runs.
- Add ETA calculation.
- Add checkpointing.
- Add idle background learning after 60 seconds.

Exact Prompt 10:

```text
Build Phase 10 for Bio-Digital. PURE MACHINE CODE FOCUS. ZERO ABSTRACTION.

Implement the autonomous Bio-Digital learn scheduler. Add `AUTO_LEARN`, `CHAT_RUN`, and idle background learning after 60 seconds of no user input. Learning must checkpoint `.bdl` at fixed intervals, estimate remaining time from measured block throughput, pause immediately during chat generation, and resume during idle. No Python, no nrlpy, no wrappers, no subprocess runtime, no AST, no IR, and no VM.
```

### Phase 11: Production Release Gates

Goal:

- Add crash recovery tests.
- Add parser fuzzing.
- Add benchmark gates.
- Add coherent chat golden prompts.
- Add release report generation.

Exact Prompt 11:

```text
Build Phase 11 for Bio-Digital. PURE MACHINE CODE FOCUS. ZERO ABSTRACTION.

Implement production release gates for Bio-Digital. Add native parser fuzzing, GGUF parser fuzzing, `.bdl` crash-recovery tests, benchmark artifacts for ingest speed, learn throughput, executed TPS/WPS, virtual sparse throughput, idle-learn behavior, and coherent chat golden prompts. Generate a native release report. No Python, no nrlpy, no wrappers, no subprocess runtime, no AST, no IR, and no VM.
```

## 14. Production Acceptance Criteria

Bio-Digital is production grade only when all of these pass:

1. `bio ingest model.gguf --out model.bdst --delta model.bdl` completes or resumes after interruption.
2. `bio learn model.bdst --hours 24` runs unattended, checkpoints, reports ETA, and writes `.bdl`.
3. `bio chat model.bdst` starts without re-ingesting the GGUF.
4. Chat uses native tokenizer, native forward pass, native sampler, and native KV cache.
5. After 60 seconds idle, background learning resumes automatically.
6. New user input pauses background learning immediately.
7. `.bdl` survives process crash and checksum verification.
8. GGUF remains read-only.
9. All hot paths are C/SIMD native.
10. No `nrlpy`, Python, llama.cpp wrapper, subprocess runtime, AST, IR, or VM is used.
