Skip to content

docs: Add schema validation feature guide#308

Closed
Copilot wants to merge 2 commits intomainfrom
copilot/explain-schema-validation-dataframely
Closed

docs: Add schema validation feature guide#308
Copilot wants to merge 2 commits intomainfrom
copilot/explain-schema-validation-dataframely

Conversation

Copy link
Contributor

Copilot AI commented Mar 26, 2026

No dedicated guide explained the schema validation pipeline in depth — only the quickstart covered basics.

New guide: docs/guides/features/schema-validation.md

  • How rules are applied — column-level (zip_code|min_length), primary key uniqueness, and custom @dy.rule() decorators, all evaluated in a single polars pass
  • Three validation methodsvalidate (strict, raises ValidationError), filter (soft, returns FailureInfo), is_valid (boolean, never raises)
  • Error taxonomySchemaError (structural: missing/wrong-dtype columns) vs ValidationError (content: rule violations); is_valid silently absorbs both
  • Cast behaviourcast=True attempts dtype coercion before rules run; failed casts surface as <column>|dtype entries in FailureInfo
  • Inspecting failures — covers all FailureInfo methods: counts, invalid, details (per-row valid/invalid/unknown breakdown), and cooccurrence_counts
  • Superfluous columns — columns absent from the schema are silently dropped from validate/filter output

Also adds the guide as the first entry in docs/guides/features/index.md.


📍 Connect Copilot coding agent with Jira, Azure Boards or Linear to delegate work to Copilot in one click without leaving your project management tool.

@codecov
Copy link

codecov bot commented Mar 26, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (72fb1a6) to head (0f794a9).

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #308   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           56        56           
  Lines         3218      3218           
=========================================
  Hits          3218      3218           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Co-authored-by: riggggo <90137935+riggggo@users.noreply.github.com>
Agent-Logs-Url: https://github.com/Quantco/dataframely/sessions/bbf5b5fb-f92b-4f77-a624-1b4b02c717b5
Copilot AI changed the title [WIP] Explain how schema validation works with Dataframely docs: Add schema validation feature guide Mar 26, 2026
@github-actions github-actions bot added the documentation Improvements or additions to documentation label Mar 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants