Skip to content

fix(bedrock): place cache point before non-PDF document blocks#2001

Open
kevmyung wants to merge 1 commit intostrands-agents:mainfrom
kevmyung:fix/cache-point-non-pdf-document
Open

fix(bedrock): place cache point before non-PDF document blocks#2001
kevmyung wants to merge 1 commit intostrands-agents:mainfrom
kevmyung:fix/cache-point-non-pdf-document

Conversation

@kevmyung
Copy link
Copy Markdown
Contributor

Description

Closes #1966

PR #1821 moved cache point injection from the last assistant message to the last user message. This causes a ValidationException when the last user message contains non-PDF document blocks (md, csv, txt, html), because Bedrock's format conversion fails to preserve the type field on a cachePoint block placed directly after a non-PDF document block.

In _inject_cache_point, instead of unconditionally appending cachePoint to the end of the content array, scan for the first non-PDF document block and insert cachePoint before it. If no non-PDF document blocks exist, the existing behavior (append at end) is preserved.

Cache effectiveness is not impacted — Anthropic's prompt caching is prefix-based, so placing cachePoint before the document block still includes the document content in the cached prefix.

Related Issues

#1966
#1821

Documentation PR

N/A

Type of Change

Bug fix

Testing

  • Added test: cache point inserted before non-PDF document block
  • Added test: cache point appended at end with PDF-only document
  • Added test: mixed PDF and non-PDF documents
  • All existing cache point tests pass
  • I ran hatch run prepare

Checklist

  • I have read the CONTRIBUTING document
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cachePoint after non-PDF document blocks causes ValidationException on Bedrock ConverseStream

1 participant