perf(nbd): zero-alloc, zero-copy synchronous dispatch

[ValentaTomas]

Summary

• Replace async goroutine-per-request NBD dispatch with a synchronous loop using mmap slices directly
• Reads: Overlay.ReadSlices returns per-block mmap slices (overlay cache or template cache), written to the NBD socket via writev (net.Buffers) — zero user-space copies, zero allocations after warmup
• Writes: Cache.WriteSlice returns a writable mmap reference; socket data is read directly into the mmap page — zero user-space copies, zero allocations
• Removes per-request: goroutines, channels, sync.Pool, write lock, WaitGroup, and shutting-down machinery

Profile context

From profiling on n2-highmem-80 nodes, NBD dispatch was the dominant allocator after the multipart upload fix (#2394):

Path	Cumulative allocs	What it was
Dispatch.Handle (write buffers)	57 TB (47%)	make([]byte, request.Length) per write
cmdRead.func1 (read buffers)	20 TB (16%)	make([]byte, length) per read

This PR eliminates both — zero allocations after the first request at each size.

How it works

Reads: Instead of allocating a buffer, copying mmap data into it, then writing to socket — we collect per-block mmap slice references and send them via writev in a single syscall.

Writes: Instead of allocating a buffer, reading socket data into it, then copying into mmap — we get a writable reference to the overlay cache mmap and read socket data directly into it.

Parallelism is preserved via FlagCanMulticonn (multiple NBD socket connections per device), rather than per-request goroutines within a connection.

Test plan

• go build ./... passes
• go vet passes
• Existing NBD tests pass (require root / nbd kernel module)
• Profile on staging to verify allocation elimination

[cursor]

PR Summary

Medium Risk
Touches the hot-path NBD I/O read/write flow and introduces lock-held writable slices, so bugs could cause data corruption, deadlocks, or kernel-visible I/O errors if alignment/length or unlock/commit handling is wrong.

Overview
Reworks NBD dispatch to avoid per-request data buffers by reading and writing directly against mmap-backed cache slices: reads now gather block-aligned slices from cache/device (Overlay.ReadSlices) and stream them straight to the NBD socket, while writes read socket data directly into a writable mmap window (Cache.WriteSlice/Overlay.WriteFrom). This updates the NBD backend interface to these slice-based APIs, shrinks the dispatch read buffer to header-sized, and adjusts test utilities and the slow-backend test to exercise slice-based reads with proper context cancellation.

^{Reviewed by Cursor Bugbot for commit f2ca90c. Bugbot is set up for automated code reviews on this repo. Configure here.}

[claude]

test comment

[claude]

Cache mutex held during socket I/O. c.mu.Lock() is acquired here and not released until done() is called in dispatch.go after the entire fp.Read loop completes (line 195). The mutex is held for the full duration of reading the write payload off the socket. With FlagCanMulticonn, multiple Dispatch goroutines share the same Provider. While one goroutine is blocked on slow socket data holding c.mu, every other connection's reads and writes stall on this lock. The old design held the lock only briefly inside WriteAt, after socket data was already buffered locally. The fix is to read socket data into a temp buffer first, then acquire the lock only to copy into and mark the mmap.

[claude]

Silent truncation for non-blockSize-aligned lengths. numBlocks := length / o.blockSize discards trailing bytes when length is not a multiple of blockSize, with no error. If a non-aligned length is passed, writeReadResponse sends fewer bytes than requested and the kernel NBD driver treats this as an I/O error. Kernel NBD requests are block-aligned in practice, but a guard returning an error for non-aligned lengths would surface any future misuse immediately.

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit f2ca90c. Configure here.}

[cursor]

Mmap slices read/written without lock protection cause data race

Medium Severity

ReadSlices calls cache.Slice() without holding any lock, and the returned mmap slice references are later written to the socket by writeSlices. Previously, Overlay.ReadAt went through Cache.ReadAt which held c.mu.RLock(), providing mutual exclusion with writes via RWMutex. Now, a concurrent WriteFrom on another connection (multiconn) can modify the same mmap pages while writeSlices is reading them, producing torn data in the NBD response.

Additional Locations (1)

• packages/orchestrator/pkg/sandbox/nbd/dispatch.go#L277-L312

 

^{Reviewed by Cursor Bugbot for commit f2ca90c. Configure here.}

[cursor]

Exclusive cache lock held during entire socket read

Medium Severity

Cache.WriteSlice acquires the exclusive write lock (c.mu.Lock()), and Overlay.WriteFrom calls io.ReadFull from a network socket before releasing it via the done callback. For writes up to 32 MB, this holds the exclusive lock for the entire duration of the socket read. The old code read socket data into a temporary buffer first, then held the lock only for a brief memcpy. This serializes all lock-dependent cache operations (Close, ExportToDiff, other WriteAt calls) across all multiconn connections during potentially slow I/O.

Additional Locations (1)

• packages/orchestrator/pkg/sandbox/block/overlay.go#L110-L121

 

^{Reviewed by Cursor Bugbot for commit f2ca90c. Configure here.}