Users/bogdanbebic/staging april#494
Open
Bogdan Bebić (bogdanbebic) wants to merge 119 commits intostagingfrom
Open
Users/bogdanbebic/staging april#494Bogdan Bebić (bogdanbebic) wants to merge 119 commits intostagingfrom
Bogdan Bebić (bogdanbebic) wants to merge 119 commits intostagingfrom
Conversation
This commit adds a GitHub Actions workflow that automatically publishes the Python schema packages to PyPI when they have a version number change that is pushed to the main branch.* Packages are published in topological order, which is for now determined by the hard-coded `def level` function in `scripts/package-versions.py`. Eventually, we also want to auto-cut a GitHub Release after a successful publish, but let's do that later. *: For now, PyPI = Overture internal CodeArtifact repo, but eventually it will be public PyPI.
Vehicle dimension selectors (height, length, weight, width) use float64 instead of float32 to match the double-precision values in the data platform. Level uses int32 instead of int16 for the same reason. Axle count stays uint8 since it's a discrete count.
Cartography types (Prominence, MinZoom, MaxZoom, SortKey) from uint8 to int32; ConfidenceScore from float32 to float64. Matches actual data ranges in Overture releases.
The mypy target only checked top-level test files via `packages/*/tests/*.py`. Subdirectory tests (model_constraint/, ref/, scoping/) were never type-checked. Switch to per-directory find so mypy discovers all test files while keeping each package's conftest.py visible (avoids duplicate module conflicts across packages). Fixes in test code: - Mapping[str, object] in assert_subset (dict invariance) - Missing return type annotations on test functions - Duplicate TestModel names within same scope - JsonDict annotations for ConfigDict json_schema_extra - type: ignore for dynamically-created .When classes
Every other package had its tests/ directory in pythonpath; cli was the only one missing, making its conftest imports rely on rootdir discovery.
These packages import from overture.schema.system without declaring it as a dependency. Works in the uv workspace but breaks isolated installs. - overture-schema-core - overture-schema-cli - overture-schema-base-theme - overture-schema-buildings-theme - overture-schema-transportation-theme
Update multiple packages' pyproject.toml files to add a maintainers list, common keywords, and a [project.urls] section (Homepage, Source, Issues). Affects core, system, cli, annex, and all theme packages under packages/overture-schema-*. This centralizes package metadata to improve discoverability, attribution, and tooling integration.
* Delete .github/workflows/copy-pr-docs-to-staging.yaml * Delete .github/workflows/copy-latest-docs-to-staging.yaml
[CHORE] Add maintainers, keywords, and URLs to pyproject
pytest-subtests merged into pytest core as of pytest 9. Update test imports from pytest_subtests.SubTests to _pytest.subtests.Subtests. uv.lock was fully re-resolved, upgrading all transitive dependencies to current versions. Notable: pytest 8.4→9.0, mypy 1.18→1.19, pdoc 15→16, ruff 0.14.0→0.14.14.
* Widen `AdminLevel` from `uint8` to `int32` to match divisions data currently being generated. * Fix Pydantic rebinding self in model constraint validator
Geometry constraint tests: replace full powerset (127 subsets of 7 geometry types) with singletons + pairs + full set (29 subsets). GeometryTypeConstraint is a set-membership check — combinatorial testing beyond pairs exercises no additional code paths. Makefile: run pytest, doctest, ruff, and mypy concurrently via make -j after a single upfront uv-sync. Added -only variants (no uv-sync dep) for the parallel invocation; standalone targets preserved with their original uv-sync dependencies.
* Treat None as absent in model constraint validation
require_any_of, require_if, and forbid_if now treat None as "not
present" for constraint purposes. Fields must be both in
model_fields_set and non-null to satisfy (or violate) a constraint.
- require_any_of: None no longer satisfies "at least one must be set"
- require_if: None no longer satisfies "must be set when condition holds"
- forbid_if: None no longer violates "must not be set when condition holds"
JSON Schema generation updated to emit {"not": {"type": "null"}}
property constraints alongside "required" assertions, via a shared
required_non_null helper in _json_schema.py. Shared predicate
_field_has_non_null_value extracted onto OptionalFieldGroupConstraint
so the check lives in one place. Also fixes a stray backtick in the
require_if error message.
* Use Python None terminology and clarify constraint docs
Use Python terminology (None) instead of null in docstrings, identifiers,
and error messages. Rewrite error messages to describe what's checked
("set to a value other than None") rather than using jargon.
Eliminates duplicated validate() and __get_pydantic_json_schema__() across CountryCodeAlpha2, HexColor, LanguageTag, NoWhitespace, SnakeCase, PhoneNumber, RegionCode, and WikidataId constraints. Each is now a thin __init__-only wrapper calling super().__init__(). PatternConstraint gains optional keyword-only description, min_length, max_length parameters for JSON Schema annotations. StringConstraint gains _raise_validation_error() to deduplicate error construction across PatternConstraint, JsonPointerConstraint, and StrippedConstraint.
The previous pattern ^(\S.*)?\S$ required at least one non-whitespace
character, rejecting empty string. The validator itself accepts empty
string ("" == "".strip()), so the JSON schema was more restrictive
than the Python validation.
New pattern ^(\S(.*\S)?)?$ matches empty string (outer group optional),
single non-whitespace chars, and strings bookended by non-whitespace.
Updated all JSON schema baselines and inline expectations.
Replace `len(errors()) > 0` with error message assertions in hex color and no-whitespace invalid tests. The weak assertions only checked that validation failed, not that the correct error was raised.
Replace 16 individual valid/invalid test methods with two parametrized tests driven by PATTERN_CONSTRAINT_CASES. Covers all 8 PatternConstraint subclasses: LanguageTag, CountryCodeAlpha2, RegionCode, WikidataId, PhoneNumber, HexColor, NoWhitespace, SnakeCase. Moved SnakeCaseConstraint tests from TestErrorHandling (where they were misplaced) into the parametrized data. Non-PatternConstraint tests remain as standalone methods: base PatternConstraint (custom pattern), StrippedConstraint, and JsonPointerConstraint (empty-string special case).
The dict branch of the Pydantic validator in BBox.__get_pydantic_core_schema__ constructed a BBox from the dict but didn't return it, silently producing None. Add the missing return statement. The test parametrize case for dict input covers this path.
Bare triple-quoted strings after NewType assignments are expression statements that Python never attaches to the NewType object, leaving __doc__ as None. Convert each to an explicit __doc__ assignment so codegen and introspection tools can read them at runtime. Same pattern DocumentedEnum uses for enum member docs.
- Add -q, --tb=short to `make test` for compact output - Set verbosity_subtests=0 to suppress per-subtest progress characters (the u/,/- markers from pytest's built-in subtests support) Signed-off-by: Seth Fitzsimmons <sethfitz@amazon.com>
OvertureFeature validator error message had two continuation
lines missing the f-prefix, so {self.__class__.__name__} was
rendered literally. Also add missing space before "and".
Signed-off-by: Seth Fitzsimmons <sethfitz@amazon.com>
Signed-off-by: John McCall <john@overturemaps.org>
…-workflow-security-concerns [SECURITY] Remove vulnerabilities from GH workflows
Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 4 to 7. - [Release notes](https://github.com/astral-sh/setup-uv/releases) - [Commits](astral-sh/setup-uv@v4...v7) --- updated-dependencies: - dependency-name: astral-sh/setup-uv dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 6. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v4...v6) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5 to 6. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v5...v6) --- updated-dependencies: - dependency-name: actions/setup-python dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [aws-actions/configure-aws-credentials](https://github.com/aws-actions/configure-aws-credentials) from 4 to 6. - [Release notes](https://github.com/aws-actions/configure-aws-credentials/releases) - [Changelog](https://github.com/aws-actions/configure-aws-credentials/blob/main/CHANGELOG.md) - [Commits](aws-actions/configure-aws-credentials@v4...v6) --- updated-dependencies: - dependency-name: aws-actions/configure-aws-credentials dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
…ctions/checkout-6 [CHORE](deps)(deps): Bump actions/checkout from 4 to 6
…ctions/setup-python-6 [CHORE](deps)(deps): Bump actions/setup-python from 5 to 6
…ws-actions/configure-aws-credentials-6 [CHORE](deps)(deps): Bump aws-actions/configure-aws-credentials from 4 to 6
Signed-off-by: John McCall <john@overturemaps.org>
…stral-sh/setup-uv-7 [CHORE](deps)(deps): Bump astral-sh/setup-uv from 4 to 7
Signed-off-by: John McCall <john@overturemaps.org>
Signed-off-by: John McCall <john@overturemaps.org>
Signed-off-by: John McCall <john@overturemaps.org>
Signed-off-by: John McCall <john@overturemaps.org>
Signed-off-by: Seth Fitzsimmons <sethfitz@amazon.com>
[FEATURE] Docs preview for schema changes
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 7 to 8. - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](actions/download-artifact@v7...v8) --- updated-dependencies: - dependency-name: actions/download-artifact dependency-version: '8' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [actions/github-script](https://github.com/actions/github-script) from 6 to 8. - [Release notes](https://github.com/actions/github-script/releases) - [Commits](actions/github-script@v6...v8) --- updated-dependencies: - dependency-name: actions/github-script dependency-version: '8' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
…ctions/github-script-8 [CHORE](deps)(deps): Bump actions/github-script from 6 to 8
…ctions/download-artifact-8 [CHORE](deps)(deps): Bump actions/download-artifact from 7 to 8
See https://docs.astral.sh/uv/concepts/resolution/#dependency-cooldowns This will prevent dependencies updated within a week from being considered as upgrade candidates. Signed-off-by: Seth Fitzsimmons <sethfitz@amazon.com>
Bogdan Bebić (bogdanbebic)
requested review from
Seth Fitzsimmons (mojodna),
Stephen Epps (stepps00) and
Victor Schappert (vcschapp)
🗺️ Schema reference docs preview is live!
Note ♻️ This preview updates automatically with each push to this PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
C. Public documentation and messaging plan
Documentation on transportation theme linear referencing was updated and improved
Building type documentation -> fixed typo.
Description
Bring staging up to dev so that it contains the pydantic changes and the changes to the python build system.
Documentation website
Update the hyperlink below to put the pull request number in.
Docs preview for this PR.