Implement streaming response optimization for non-Next.js publisher proxy

[aram356]

Context

The publisher proxy currently buffers the entire response body in memory before sending any bytes to the client. For a 222KB HTML page, peak memory is ~4x the response size and no bytes reach the client until all processing completes.

Spec

See streaming response design spec (PR #562).

Plan

See implementation plan (PR #562).

Phase 1: Make streaming pipeline chunk-emitting (PR #583)

Ships independently with immediate memory savings.

• Phase 1, Task 1: Fix encoder finalization in process_through_compression #568 — Phase 1, Task 1: Fix encoder finalization in process_through_compression
• Phase 1, Task 2: Convert process_gzip_to_gzip to chunk-based processing #569 — Phase 1, Task 2: Convert process_gzip_to_gzip to chunk-based processing
• Phase 1, Task 3: Convert decompress_and_process to chunk-based processing #570 — Phase 1, Task 3: Convert decompress_and_process to chunk-based processing
• Phase 1, Task 4: Rewrite HtmlRewriterAdapter for incremental streaming #571 — Phase 1, Task 4: Rewrite HtmlRewriterAdapter for incremental streaming
• Phase 1, Task 5: Full verification #572 — Phase 1, Task 5: Full verification

Phase 2: Stream responses to client via StreamingBody (PR #585)

Depends on Phase 1. Adds TTFB/TTLB improvement.

• Phase 2, Task 6: Migrate entry point from #[fastly::main] to raw main() #573 — Phase 2, Task 6: Migrate entry point from #[fastly::main] to raw main()
• Phase 2, Task 7: Refactor process_response_streaming to accept W: Write #574 — Phase 2, Task 7: Refactor process_response_streaming to accept W: Write
• Phase 2, Task 8: Add streaming path to publisher proxy #575 — Phase 2, Task 8: Add streaming path to publisher proxy
• Phase 2, Task 9: Full verification #576 — Phase 2, Task 9: Full verification
• Phase 2, Task 10: Chrome DevTools MCP baseline and comparison #577 — Phase 2, Task 10: Chrome DevTools MCP baseline and comparison

Phase 3: Make script rewriters fragment-safe

Depends on Phase 2. Removes the buffered fallback, enabling full streaming even with GTM/NextJS script rewriters active.

• Phase 3, Task 11: Make NextJsNextDataRewriter fragment-safe #586 — Phase 3, Task 11: Make NextJsNextDataRewriter fragment-safe
• Phase 3, Task 12: Make GoogleTagManagerIntegration rewrite fragment-safe #587 — Phase 3, Task 12: Make GoogleTagManagerIntegration rewrite fragment-safe
• Phase 3, Task 13: Remove buffered mode from HtmlRewriterAdapter #588 — Phase 3, Task 13: Remove buffered mode from HtmlRewriterAdapter
• Phase 3, Task 14: Always use streaming adapter in create_html_processor #589 — Phase 3, Task 14: Always use streaming adapter in create_html_processor
• Phase 3, Task 15: Full verification and regression tests #590 — Phase 3, Task 15: Full verification and regression tests

Acceptance Criteria

• Streaming activates when Next.js is disabled and backend returns 2xx
• Peak memory per request reduced from ~4x to constant (chunk buffer + parser state)
• Client receives first body bytes after first processed chunk, not after full buffering
• No regressions on static, auction, or discovery endpoints
• Buffered fallback works correctly when post-processors are registered
• (Phase 3) Streaming works even with GTM/NextJS script rewriters active