Skip to content

itest: Add reusable integration test library crate#231

Open
cgwalters wants to merge 1 commit intobootc-dev:mainfrom
cgwalters:itest
Open

itest: Add reusable integration test library crate#231
cgwalters wants to merge 1 commit intobootc-dev:mainfrom
cgwalters:itest

Conversation

@cgwalters
Copy link
Collaborator

Extract the test infrastructure pattern shared across bcvk, ostree, composefs-rs, and bootc into a standalone library crate. This follows the direction outlined in bootc-dev/infra repository-structure.md to reduce code duplication for integration testing.

The crate provides:

  • Test registration via linkme distributed slices and integration_test! / parameterized_integration_test! macros (no manual test lists)
  • Privilege dispatch via privileged_test! and booted_test! macros that auto-dispatch to bcvk VMs when not running as root
  • JUnit XML output via quick-junit (set JUNIT_OUTPUT env var)
  • A harness runner (run_tests / run_tests_with_config) that ties together libtest-mimic, parameterized test expansion, and JUnit

The integration-tests crate is updated to use itest instead of its own hand-rolled infrastructure. Test function return types are migrated from color_eyre::Result to anyhow::Result to align with the library standard (the tests never used color_eyre-specific features).

Assisted-by: OpenCode (Claude Opus 4.6)

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

The pull request introduces a new itest crate for integration testing, featuring automatic test registration, privilege tiers with VM dispatch, and JUnit XML output. The itest crate provides macros for registering tests and supports running tests that require root privileges by dispatching them to a bcvk VM. Review comments suggest improving the error message in the test harness, using a more robust method for creating test name suffixes, and addressing a potential panic in the privilege dispatch logic.

let outcome = TestOutcome {
name,
duration,
result: result.as_ref().map(|_| ()).map_err(|e| format!("{e:?}")),

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Consider adding a more descriptive error message to the map_err closure to provide better context for test failures.

Suggested change
result: result.as_ref().map(|_| ()).map_err(|e| format!("{e:?}")),
result.map_err(|e| format!("Test {} failed: {:?}", name, e)).into()

Comment on lines +192 to +193
pub fn image_to_test_suffix(image: &str) -> String {
image.replace(|c: char| !c.is_alphanumeric(), "_")

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Consider using a more robust method for creating test name suffixes, such as URL-safe base64 encoding, to avoid potential collisions and ensure compatibility with various filesystems.

@cgwalters cgwalters marked this pull request as draft March 20, 2026 22:22
@henrywang
Copy link

Thanks @cgwalters!!
We can land this and test it in ostreedev/ostree#3576.

@cgwalters cgwalters force-pushed the itest branch 2 times, most recently from d2a944a to 7d58ca1 Compare March 23, 2026 12:57
@cgwalters cgwalters marked this pull request as ready for review March 23, 2026 13:09
@cgwalters cgwalters enabled auto-merge (rebase) March 23, 2026 13:09
We were growing too much duplication between projects around
integration tests. Extract a shared helper library that we'll
vendor via git crate dependencies.

Assisted-by: OpenCode (Claude Opus 4.6)
Signed-off-by: Colin Walters <walters@verbum.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants