Skip to content

Phase 4: Stream binary pass-through responses via io::copy#594

Open
aram356 wants to merge 5 commits intofeature/streaming-pipeline-phase3from
feature/streaming-pipeline-phase4
Open

Phase 4: Stream binary pass-through responses via io::copy#594
aram356 wants to merge 5 commits intofeature/streaming-pipeline-phase3from
feature/streaming-pipeline-phase4

Conversation

@aram356
Copy link
Copy Markdown
Collaborator

@aram356 aram356 commented Mar 27, 2026

Summary

Stream non-processable 2xx responses (images, fonts, video) directly to the client instead of buffering the entire body in memory. Uses send_to_client() with the unmodified body to preserve Content-Length and avoid chunked encoding overhead.

Closes #592, closes #593.
Part of epic #563. Depends on Phase 3 (#591).

Performance results (staging vs production, median over 8 runs, Chrome 1440x900)

Metric Production (v135, buffered) Staging (v141, streaming+passthrough) Delta
TTFB 57 ms 28 ms -29 ms (-51%)
First Paint 252 ms 182 ms -70 ms (-28%)
First Contentful Paint 252 ms 182 ms -70 ms (-28%)
DOM Content Loaded 322 ms 248 ms -74 ms (-23%)
DOM Complete 678 ms 582 ms -96 ms (-14%)

Measured on getpurpose.ai via Chrome DevTools Protocol with --host-resolver-rules to route to staging Fastly edge.

How it works

The streaming gate in handle_publisher_request now has three outcomes:

is_success (2xx, not 204)
├── should_process && (!is_html || !has_post_processors) → Stream (pipeline)
├── should_process && is_html && has_post_processors     → Buffered (post-processors)
└── !should_process                                      → PassThrough (send_to_client)

204 No Content or !is_success
└── any content type                                     → Buffered

PassThrough reattaches the unmodified body and sends via send_to_client(), preserving Content-Length. This avoids both WASM memory buffering and chunked encoding overhead.

What changed

File Lines What
publisher.rs +60 -5 PassThrough variant, gate logic, 204 exclusion, tests
main.rs +5 Handle PassThrough: reattach body, return for send_to_client()

Tests added

  • pass_through_gate_streams_non_processable_2xx — images/fonts return PassThrough
  • pass_through_gate_buffers_non_processable_error — 4xx stays Buffered
  • pass_through_gate_does_not_apply_to_processable_content — HTML goes through Stream
  • pass_through_gate_excludes_204_no_content — 204 stays Buffered (HTTP spec)
  • pass_through_gate_applies_with_empty_request_host — empty host still passes through
  • pass_through_preserves_body_and_content_length — byte-for-byte identity + CL preserved

Verification

  • cargo test --workspace — 772 passed, 0 failed
  • cargo clippy --workspace --all-targets --all-features -- -D warnings — clean
  • cargo fmt --all -- --check — clean
  • cargo build --release --target wasm32-wasip1 — success

Test plan

  • Pass-through gate unit tests pass (6 tests)
  • Byte-level body + Content-Length preservation test passes
  • 204 No Content exclusion test passes
  • All existing streaming/buffered tests pass
  • Full workspace tests pass (772)
  • WASM build succeeds
  • Staging performance verified (51% TTFB, 28% FCP, 14% DOM Complete improvement)

aram356 added 5 commits March 27, 2026 12:48
Non-processable 2xx responses (images, fonts, video) now stream
directly to the client via PublisherResponse::PassThrough instead
of buffering the entire body in memory. Content-Length is preserved
since the body is unmodified.
Tests verify non-processable 2xx responses return PassThrough,
non-processable errors stay Buffered, and processable content
goes through Stream (not PassThrough).
Adds pass_through_preserves_body_and_content_length test that
verifies io::copy produces identical output and Content-Length
is preserved. Updates handle_publisher_request doc to describe
all three response variants.
- Exclude 204 No Content from PassThrough (must not have body)
- Remove Content-Length before streaming (stream_to_client uses
  chunked encoding, keeping both violates HTTP spec)
- Add tests for 204 exclusion and empty-host interaction
- Update doc comment and byte-level test to reflect CL removal
PassThrough reattaches the unmodified body and uses send_to_client()
instead of stream_to_client() + io::copy. This preserves
Content-Length (avoids chunked encoding overhead for images/fonts)
and lets Fastly stream from its internal buffer without WASM memory
buffering.
@aram356 aram356 self-assigned this Mar 27, 2026
@aram356 aram356 marked this pull request as draft March 27, 2026 21:14
@aram356 aram356 marked this pull request as draft March 27, 2026 21:14
@aram356 aram356 marked this pull request as ready for review March 27, 2026 23:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Phase 4, Task 17: Binary pass-through tests and verification Phase 4, Task 16: Stream binary pass-through responses via io::copy

1 participant