Skip to content

fix: truncate nanosecond-precision timestamps for BigQuery before casting#989

Open
devin-ai-integration[bot] wants to merge 1 commit intomasterfrom
devin/1776323745-fix-bigquery-nanosecond-timestamp
Open

fix: truncate nanosecond-precision timestamps for BigQuery before casting#989
devin-ai-integration[bot] wants to merge 1 commit intomasterfrom
devin/1776323745-fix-bigquery-nanosecond-timestamp

Conversation

@devin-ai-integration
Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration bot commented Apr 16, 2026

Summary

Adds a bigquery__edr_cast_as_timestamp dispatch macro that truncates fractional seconds beyond microsecond precision (6 digits) before casting string values to TIMESTAMP.

Problem: Starting with dbt-bigquery >= 1.11, nanosecond-precision timestamps (9 fractional digits, e.g. 2026-04-03T10:50:50.961498756Z) began appearing in string columns like execute_completed_at in dbt_run_results. BigQuery's TIMESTAMP type only supports microsecond precision (6 digits), so CAST(... AS TIMESTAMP) fails with Invalid timestamp errors — completely blocking edr report and edr send-report. Reported by multiple users in the community Slack.

Fix: Uses regexp_replace to truncate to 6 fractional digits before casting, following the same pattern as the existing Dremio override. Timestamps with ≤6 fractional digits or no fractional seconds pass through unchanged. The existing bigquery__edr_cast_as_date macro already calls edr_cast_as_timestamp internally, so it also benefits.

Review & Testing Checklist for Human

  • Verify BigQuery regex syntax: confirm r'(\.\d{6})\d+' with '\\1' backreference is valid BigQuery REGEXP_REPLACE usage
  • Test against a real BigQuery instance with nanosecond-precision timestamps in dbt_run_results — CI likely does not cover BigQuery-specific behavior
  • Verify edge cases: timestamps with exactly 6 fractional digits, no fractional seconds, and NULL values should all cast correctly without errors

Recommended test plan: Run edr report or edr send-report against a BigQuery project that has nanosecond-precision timestamps in the dbt_run_results table (or manually insert a row with e.g. 2026-04-03T10:50:50.961498756Z in execute_completed_at) and confirm the report generates successfully.

Notes

  • This only affects BigQuery; all other adapters continue using the default cast(... as timestamp) path
  • The regex is a no-op for timestamps that already have ≤6 fractional digits, so there is no behavioral change for well-formed data

Link to Devin session: https://app.devin.ai/sessions/8841375f9b774c8088e1bff6c8e44f52
Requested by: @MikaKerman

Summary by CodeRabbit

  • Bug Fixes
    • Resolved timestamp casting issues for BigQuery when handling high-precision values with fractional seconds, ensuring accurate data type conversions.

…ting

BigQuery TIMESTAMP only supports microsecond precision (6 fractional digits).
Starting with dbt-bigquery >= 1.11, nanosecond-precision timestamps (9 fractional
digits) may appear in string columns like execute_completed_at. This causes
'Invalid timestamp' errors when edr_cast_as_timestamp casts these values.

Add a bigquery__edr_cast_as_timestamp dispatch that truncates fractional seconds
beyond 6 digits before casting, matching the pattern used by the existing Dremio
override.

Co-Authored-By: mika@elementary-data.com <mika.kerman@gmail.com>
@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@github-actions
Copy link
Copy Markdown
Contributor

👋 @devin-ai-integration[bot]
Thank you for raising your pull request.
Please make sure to add tests and document all user-facing changes.
You can do this by editing the docs files in the elementary repository.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 16, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 33843bd0-d2f7-4df9-909a-2fc8ff7f9037

📥 Commits

Reviewing files that changed from the base of the PR and between 9f32d47 and 760e97a.

📒 Files selected for processing (1)
  • macros/utils/data_types/cast_column.sql

📝 Walkthrough

Walkthrough

A BigQuery-specific macro override is added to cast_column.sql that handles timestamp casting by converting to string, truncating fractional seconds to 6 digits using regex, and casting to the elementary timestamp type.

Changes

Cohort / File(s) Summary
BigQuery Timestamp Casting
macros/utils/data_types/cast_column.sql
Added bigquery__edr_cast_as_timestamp macro that converts timestamps to strings, truncates fractional seconds to 6 digits via regexp_replace, and casts to the appropriate type.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐰 In BigQuery's grand temporal halls,
A macro hops through fractional walls,
Six digits we keep, the rest we discard,
Truncating precision—no task is too hard!
Timestamps now dance in elastic delight.

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: adding a BigQuery-specific macro that truncates nanosecond-precision timestamps before casting. It directly reflects the core fix in the changeset.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch devin/1776323745-fix-bigquery-nanosecond-timestamp

Comment @coderabbitai help to get the list of available commands and usage tips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant