Sanity tests: dynamic stack setup, report context, fixes, security cleanup#502
Open
AniketDev7 wants to merge 36 commits intodevelopmentfrom
Open
Sanity tests: dynamic stack setup, report context, fixes, security cleanup#502AniketDev7 wants to merge 36 commits intodevelopmentfrom
AniketDev7 wants to merge 36 commits intodevelopmentfrom
Conversation
- Rewrite API tests for comprehensive SDK coverage (487 tests) - Add 2FA/TOTP authentication test cases - Add test utilities for request logging, assertions, and cleanup - Implement stack cleanup using direct API calls - Add complex mock schemas from exported CDA stack - Add test:sanity-nocov script for Node.js v22 compatibility - Fix test reliability with proper delays and error handling - Remove obsolete test files and unused mock data
- Add sanity-check-backup/ to gitignore - Add .vscode/ to gitignore - Remove env.example.txt (credentials should be managed separately)
- Improve authentication handling for bulk job status API - Add better error handling for branch creation - Skip dependent tests gracefully if resource creation fails - Increase wait time after branch creation for API propagation
- Asset, Release, and Workflow tests now fetch environment from testData - Fallback to querying API if testData not available - Prevents failures when environment names include timestamps
- Fix publish rules to use correct SDK method (workflow().publishRule()) - Make workflow, asset, and release tests self-contained by creating temp environments if needed - Increase timeouts for global field and asset tests - Preserve user-created management tokens in cleanup (only delete test-created ones) - Improve webhook cleanup with sequential deletion and logging - Use shorter environment names (max 10 chars)
Add comprehensive test coverage for asset_fields[] parameter in Entry API: - Fetch with single/multiple asset_fields values - Query with single/multiple asset_fields values - Combined with other query params (locale, include_workflow, etc.) - Edge case: empty asset_fields array - All 4 supported values: user_defined_fields, embedded, ai_suggested, visual_markups Note: Tests are disabled by default. Set DAM_2_0_ENABLED=true in .env to enable once the AM 2.0 feature is available in the test environment.
- Dynamic stack/token setup; Expected vs Actual + cURL in Mochawesome reports - ContentstackClient: use instrumented client by default, new client when authtoken passed - Fix token validation assertions, audit log expected status, MEMBER_EMAIL usage - Security: replace blt UIDs and testcs@contentstack.com with placeholders - Update .talismanrc checksums for modified sanity test files
- Refactor passed-test context: compute Expected vs Actual once and add in single place - Prevents missing block when cURL/API Request are present (e.g. organization teams) - Use nullish coalescing for lastRequest fields to avoid undefined in output - Add test-curls.txt to .gitignore
- Add trackedExpect import and key success-path assertions in: globalfield, branch, bulkOperation, entryVariants, terms, ungroupedVariants, variants, contentType, branchAlias, taxonomy, previewToken, team, webhook, variantGroup, token, environment, extension, label, role - Mochawesome report now shows specific Expected vs Actual in Assertions Verified for easier debugging - Update .talismanrc checksums for modified sanity files
Unit tests (test/unit/mock/objects.js) import singlepageCT from ../../sanity-check/mock/content-type. Add content-type.js so test:unit:report:json runs and report.json is generated in CI.
- asset-test.js: fix trailing spaces, remove unused uploadedAssetUid, add no-unused-expressions disables for Chai expect() (lint check) - Add test/sanity-check/mock/customUpload.html and upload.html so unit tests (asset-test, concurrency-Queue-test) find expected files and Build & Test passes (622 passes, 0 failures)
|
Coverage report for commit: db467ff Summary - Lines: 82.41% | Methods: 95.88% | Branches: 65.60%
🤖 comment via lucassabreu/comment-coverage-clover |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
- branch-test: remove unused mock imports and createdBranch; fix trailing spaces, padded-blocks - auditlog-test: remove unused testData; fix trailing spaces, padded-blocks, no-unused-expressions
- Remove unused shortId import - Fix trailing spaces, padded-blocks (via eslint --fix) - Add no-unused-expressions disables for Chai expect()
Chai expect() triggers no-unused-expressions in Standard. Override for test/**/*.js so test files don't need eslint-disable on every expect().
- Remove unused testData import - Fix trailing spaces, prefer-const (jobIds) via eslint --fix - Update .talismanrc checksum for bulkOperation-test.js
- Add trackedExpect to sanity API tests for Mochawesome expected/actual reporting - Fix no-unused-vars, no-undef across sanity-check API tests and helpers - Remove unused imports and variables (contentType, entry, role, workflow, etc.) - Fix no-return-await in role-test; add after/before to stack-test and team-test - ESLint: no-useless-escape off for test/**; promise/param-names fix in testSetup - Remove unused formatValueCompact and headersToCurl from testHelpers; sanity.js import cleanup - Update .talismanrc checksums for modified sanity test files
- environment-test: use separate temp env for 'update name' test so shared development env is never renamed; bulk/entry/release/workflow keep using testData.environments.development - bulkOperation-test: use testData.environments.development.name (envName) instead of hardcoded 'development' for all publish/unpublish payloads - .talismanrc: update checksums for modified test files and merge artifacts - No sensitive data: only process.env.API_KEY and env var references
🔒 Security Scan Results
⏱️ SLA Breach Summary
✅ BUILD PASSED - All security checks passed |
🔒 Security Scan Results
⏱️ SLA Breach Summary
✅ BUILD PASSED - All security checks passed |
🔒 Security Scan Results
⏱️ SLA Breach Summary
✅ BUILD PASSED - All security checks passed |
…improve logging. Adjusted maxAttempts for waitForJobReady function from 10 to 15 and modified delay from 2000ms to 5000ms. Updated test cases to reflect these changes and ensure consistency in job status checks. Updated checksum in .talismanrc for bulkOperation-test.js.
🔒 Security Scan Results
⏱️ SLA Breach Summary
✅ BUILD PASSED - All security checks passed |
🔒 Security Scan Results
⏱️ SLA Breach Summary
✅ BUILD PASSED - All security checks passed |
harshithad0703
previously approved these changes
Mar 12, 2026
AniketDev7
dismissed
harshithad0703’s stale review
The merge-base changed after approval.
harshithad0703
previously approved these changes
Mar 12, 2026
🔒 Security Scan Results
⏱️ SLA Breach Summary
✅ BUILD PASSED - All security checks passed |
1 similar comment
🔒 Security Scan Results
⏱️ SLA Breach Summary
✅ BUILD PASSED - All security checks passed |
AniketDev7
force-pushed
the
feat/dam-2.0-test-cases
branch
from
9cd59a3 to
67bc5d5
Compare
🔒 Security Scan Results
⏱️ SLA Breach Summary
✅ BUILD PASSED - All security checks passed |
AniketDev7
force-pushed
the
feat/dam-2.0-test-cases
branch
from
67bc5d5 to
e56f4c2
Compare
🔒 Security Scan Results
⏱️ SLA Breach Summary
✅ BUILD PASSED - All security checks passed |
…ob timeouts - OAuth: Compute Developer Hub URL from HOST env variable in test code - Handles environment-specific URL transformations without SDK changes - All OAuth tests have 15s timeout for network calls - Refactored to reuse global authtoken (avoids token limit) - Asset: Conditional description assertion for API version compatibility - BulkOperation: Add 90s timeout to 13 job status tests (waitForJobReady takes ~75s max)
AniketDev7
force-pushed
the
feat/dam-2.0-test-cases
branch
from
e56f4c2 to
33db55b
Compare
🔒 Security Scan Results
⏱️ SLA Breach Summary
✅ BUILD PASSED - All security checks passed |
- Reduced maxAttempts from 15 to 8 - Reduced delay from 5s to 3s - Total max polling time: 24s (was 75s) - Test timeout: 30s (was 90s) - Faster test execution now that bulk job route is fixed
🔒 Security Scan Results
⏱️ SLA Breach Summary
✅ BUILD PASSED - All security checks passed |
1 similar comment
🔒 Security Scan Results
⏱️ SLA Breach Summary
✅ BUILD PASSED - All security checks passed |
- Max polling attempts: 5 (reduced from 15) - Delay between polls: 5s (increased from 3s) - Initial wait: 15s (kept unchanged) - Test timeout: 30s (reduced from 90s) - Max polling time: 25s (5 attempts × 5s) - Total max time per test: ~40s
AniketDev7
force-pushed
the
feat/dam-2.0-test-cases
branch
from
6219718 to
63a0140
Compare
🔒 Security Scan Results
⏱️ SLA Breach Summary
✅ BUILD PASSED - All security checks passed |
OAuth Changes: - Revert to separate login for OAuth tests (creates authtoken #2) - Maintains proper business logic separation between basic auth and OAuth flow - Total: 2 authtokens (both reused, no token limit issue) Bulk Operation Changes: - Expose 401/403 authentication errors immediately (don't retry) - Previously: silently caught ALL errors, hiding auth failures - Now: fail fast on auth errors, retry only on network/5xx errors - Increased polling: 5→10 attempts (50s max), 60s timeout - Result: tests will show real error instead of generic timeout This fixes the 401 errors visible in pipeline console that were being masked.
🔒 Security Scan Results
⏱️ SLA Breach Summary
✅ BUILD PASSED - All security checks passed |
Fix 401 Unauthorized errors in bulk operation tests by using the dynamically created stack's API key from testSetup.testContext instead of process.env.API_KEY. The helper functions (doBulkOperation, doBulkOperationWithManagementToken, makeManagementToken) were using process.env.API_KEY which wasn't set at the time of execution, causing authentication failures on both dev9 and dev11 environments. Changes: - Updated helper functions to use testSetup.testContext.stackApiKey - Ensures correct API key is used for all bulk operations - Fixes job status check failures and other bulk operation 401 errors - Updated .talismanrc checksum for bulkOperation-test.js
🔒 Security Scan Results
⏱️ SLA Breach Summary
✅ BUILD PASSED - All security checks passed |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Updates CMA SDK sanity tests with dynamic stack setup, richer reporting, reliability fixes, and security hardening.
Changes-