Neural Fabric v2: What Changed and Why
Neural Fabric v1 shipped in 2022. It worked well in data centers. It was mediocre at the edge. The memory allocator assumed large contiguous buffers. The scheduler assumed constant-latency compute. The sharding logic assumed high-bandwidth interconnects.
None of these hold on edge hardware.
The Three Major Rewrites
1. Memory Allocator
Edge chips have fragmented memory hierarchies: L1 scratch RAM (32KB), PSRAM (8MB), and flash (variable latency). v1's single-pool allocator caused constant cache thrashing.
v2 uses a tier-aware allocator that tracks per-layer access patterns during a calibration run and assigns each tensor to the appropriate memory tier. Cache miss rate dropped 78%.
2. Scheduler
v1 used a static execution graph compiled ahead of time. This is fast but brittle — any deviation from expected latency cascades into stalls.
v2 uses a dynamic scheduler with latency feedback. Each operator reports its actual execution time, and the scheduler adjusts the ready queue dynamically. This adds 0.4ms overhead but eliminates 94% of tail-latency events from unexpected stalls.
3. Heterogeneous Sharding
Model sharding across a cloud TPU and an edge NPU requires understanding that the two halves are not in a symmetric relationship. The edge half must be self-sufficient — it cannot wait for the cloud to complete its forward pass.
v2 introduces partial completion sharding: the model is split at a point where the edge half can produce a useful (lower-quality) output independently, while the cloud half enriches that output when available. QoS degrades gracefully under network partitions.