New Feature: /Application function / test-spec gap analysis#191
Open
gavinbaumanis wants to merge 30 commits intoOrtus-Solutions:developmentfrom
Open
New Feature: /Application function / test-spec gap analysis#191gavinbaumanis wants to merge 30 commits intoOrtus-Solutions:developmentfrom
gavinbaumanis wants to merge 30 commits intoOrtus-Solutions:developmentfrom
Conversation
When running testbox on boxlang CLI, no `cgi` scope exists. Thus we need to fall back to an empty string for the `hostname` junit attribute.
Bumps [taiki-e/create-gh-release-action](https://github.com/taiki-e/create-gh-release-action) from 1.9.1 to 1.9.2. - [Release notes](https://github.com/taiki-e/create-gh-release-action/releases) - [Changelog](https://github.com/taiki-e/create-gh-release-action/blob/main/CHANGELOG.md) - [Commits](taiki-e/create-gh-release-action@v1.9.1...v1.9.2) --- updated-dependencies: - dependency-name: taiki-e/create-gh-release-action dependency-version: 1.9.2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 7 to 8. - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](actions/download-artifact@v7...v8) --- updated-dependencies: - dependency-name: actions/download-artifact dependency-version: '8' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 6 to 7. - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](actions/upload-artifact@v6...v7) --- updated-dependencies: - dependency-name: actions/upload-artifact dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
…into development
…into development
…ge is implemented.
- MetadataSmokeService + MetadataSmokeReporter + Simple embed
…is and smoke test functionality AND align the buttons in the headers - for all functionalities.
… on ACF/Lucee, getClassMetadata on BoxLang; encodeForHTML in runner/reports
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tests have been added and all pass.
Description
I realise you may not like / want what I have added in here.
There are 3 features that I have added together in this PR.
Gap Analysis:
Gap analysis is a heuristic TestBox mode that compares public/remote function names taken from component metadata against a single blob of test/spec text (the contents of .cfc/.cfm files under the configured test directory paths, respecting recurse like a normal directory run). A function is treated as “covered” when its lowercased name appears as a substring somewhere in that corpus. The component prefix is inferred from coveragePathToCapture and application mappings (longest mapping match).
It is not line coverage, not proof that a test calls or asserts that function, and not metadata smoke (which validates components load and reflect cleanly). False positives and negatives are possible (name collisions, commented-out strings, indirect coverage, etc.).
Typical inputs: same runner URL as a normal test run, plus gapAnalysis=true, a non-empty coveragePathToCapture (source root), and directory (test roots). The HTML runner skips normal testbox.run() and renders an HTML report using the same styling family as the Simple reporter.
Use case: a coarse signal that test/spec text at least mentions function names from application CFCs. TO be used as a test-gap hint, not as a quality gate on its own.
Smoke Test:
Metadata smoke is an optional TestBox mode that loads components by dotted path and runs lightweight reflection checks on public/remote functions (for example that metadata is readable and looks structurally sane).
Smoke test with dummy-invoke:
Invoke those methods with synthetic arguments for required parameters only; invoke failures are swallowed so the test run reports what could be exercised, not full behavioural correctness.
It is not line coverage, not execution coverage, and not the same as gap analysis (which compares function names to test file text). It does not prove that real tests exist or that business logic is correct.
Typical inputs: a directory walk (root + dotted prefix + optional excludes), or a single component. The HTML runner exposes these via URL parameters (e.g. metadataSmoke=true plus directory, or component keys).
Use case: a fast sanity pass that components still load and expose expected metadata across a tree—useful after refactors, mapping changes, or large merges—without running the full unit suite.
I have added a button in the header for these, too.
Jira Issues
All PRs must have an accompanied Jira issue. Please make sure you created it and linked it here.
Type of change
Please delete options that are not relevant.
Checklist