Skip to content

New Feature: /Application function / test-spec gap analysis#191

Open
gavinbaumanis wants to merge 30 commits intoOrtus-Solutions:developmentfrom
gavinbaumanis:feature/gap-analysis
Open

New Feature: /Application function / test-spec gap analysis#191
gavinbaumanis wants to merge 30 commits intoOrtus-Solutions:developmentfrom
gavinbaumanis:feature/gap-analysis

Conversation

@gavinbaumanis
Copy link
Copy Markdown
Contributor

@gavinbaumanis gavinbaumanis commented Mar 27, 2026

Tests have been added and all pass.

Description

I realise you may not like / want what I have added in here.
There are 3 features that I have added together in this PR.

Gap Analysis:
Gap analysis is a heuristic TestBox mode that compares public/remote function names taken from component metadata against a single blob of test/spec text (the contents of .cfc/.cfm files under the configured test directory paths, respecting recurse like a normal directory run). A function is treated as “covered” when its lowercased name appears as a substring somewhere in that corpus. The component prefix is inferred from coveragePathToCapture and application mappings (longest mapping match).

It is not line coverage, not proof that a test calls or asserts that function, and not metadata smoke (which validates components load and reflect cleanly). False positives and negatives are possible (name collisions, commented-out strings, indirect coverage, etc.).

Typical inputs: same runner URL as a normal test run, plus gapAnalysis=true, a non-empty coveragePathToCapture (source root), and directory (test roots). The HTML runner skips normal testbox.run() and renders an HTML report using the same styling family as the Simple reporter.

Use case: a coarse signal that test/spec text at least mentions function names from application CFCs. TO be used as a test-gap hint, not as a quality gate on its own.

Smoke Test:
Metadata smoke is an optional TestBox mode that loads components by dotted path and runs lightweight reflection checks on public/remote functions (for example that metadata is readable and looks structurally sane).

Smoke test with dummy-invoke:
Invoke those methods with synthetic arguments for required parameters only; invoke failures are swallowed so the test run reports what could be exercised, not full behavioural correctness.

It is not line coverage, not execution coverage, and not the same as gap analysis (which compares function names to test file text). It does not prove that real tests exist or that business logic is correct.

Typical inputs: a directory walk (root + dotted prefix + optional excludes), or a single component. The HTML runner exposes these via URL parameters (e.g. metadataSmoke=true plus directory, or component keys).

Use case: a fast sanity pass that components still load and expose expected metadata across a tree—useful after refactors, mapping changes, or large merges—without running the full unit suite.

I have added a button in the header for these, too.

Jira Issues

All PRs must have an accompanied Jira issue. Please make sure you created it and linked it here.

Bug Tracker: #190

Type of change

Please delete options that are not relevant.

  • New Feature

Checklist

  • My code follows the style guidelines of this project cfformat
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

lmajano and others added 30 commits April 2, 2025 00:05
When running testbox on boxlang CLI, no `cgi` scope exists. Thus we need to fall back to an empty string for the `hostname` junit attribute.
Bumps [taiki-e/create-gh-release-action](https://github.com/taiki-e/create-gh-release-action) from 1.9.1 to 1.9.2.
- [Release notes](https://github.com/taiki-e/create-gh-release-action/releases)
- [Changelog](https://github.com/taiki-e/create-gh-release-action/blob/main/CHANGELOG.md)
- [Commits](taiki-e/create-gh-release-action@v1.9.1...v1.9.2)

---
updated-dependencies:
- dependency-name: taiki-e/create-gh-release-action
  dependency-version: 1.9.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 7 to 8.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](actions/download-artifact@v7...v8)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-version: '8'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 6 to 7.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](actions/upload-artifact@v6...v7)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
- MetadataSmokeService + MetadataSmokeReporter + Simple embed
…is and smoke test functionality AND align the buttons in the headers - for all functionalities.
… on ACF/Lucee, getClassMetadata on BoxLang; encodeForHTML in runner/reports
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants