CLDSRV-872: store checksum in object metadata by leif-scality · Pull Request #6114 · scality/cloudserver

leif-scality · 2026-03-17T15:49:59Z

No description provided.

bert-e · 2026-03-17T15:50:04Z

Hello leif-scality,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options

name	description	privileged	authored
`/after_pull_request`	Wait for the given pull request id to be merged before continuing with the current one.
`/bypass_author_approval`	Bypass the pull request author's approval	⭐
`/bypass_build_status`	Bypass the build and test status	⭐
`/bypass_commit_size`	Bypass the check on the size of the changeset `TBA`	⭐
`/bypass_incompatible_branch`	Bypass the check on the source branch prefix	⭐
`/bypass_jira_check`	Bypass the Jira issue check	⭐
`/bypass_peer_approval`	Bypass the pull request peers' approval	⭐
`/bypass_leader_approval`	Bypass the pull request leaders' approval	⭐
`/approve`	Instruct Bert-E that the author has approved the pull request.		✍️
`/create_pull_requests`	Allow the creation of integration pull requests.
`/create_integration_branches`	Allow the creation of integration branches.
`/no_octopus`	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
`/unanimity`	Change review acceptance criteria from `one reviewer at least` to `all reviewers`
`/wait`	Instruct Bert-E not to run until further notice.

Available commands

name	description	privileged
`/help`	Print Bert-E's manual in the pull request.
`/status`	Print Bert-E's current status in the pull request `TBA`
`/clear`	Remove all comments from Bert-E from the history `TBA`
`/retry`	Re-start a fresh build `TBA`
`/build`	Re-start a fresh build `TBA`
`/force_reset`	Delete integration branches & pull requests, and restart merge process from the beginning.
`/reset`	Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

bert-e · 2026-03-17T15:50:11Z

Incorrect fix version

The Fix Version/s in issue CLDSRV-872 contains:

None

Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:

9.4.0

Please check the Fix Version/s of CLDSRV-872, or the target
branch of this pull request.

codecov · 2026-03-17T15:54:30Z

❌ 2 Tests Failed:

Tests completed	Failed	Passed	Skipped
8450	2	8448	0

View the top 2 failed test(s) by shortest run time

"after each" hook for "should replicate/put metadata to a destination that has a suspended null version with internal version"::backbeat routes for replication (cross account) "after each" hook for "should replicate/put metadata to a destination that has a suspended null version with internal version"

Stack Traces | 0.067s run time

The bucket you tried to delete is not empty.

"after each" hook for "should replicate/put metadata to a destination that has a null version"::backbeat routes for replication (same account) "after each" hook for "should replicate/put metadata to a destination that has a null version"

Stack Traces | 0.097s run time

The bucket you tried to delete is not empty.

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

package.json

lib/api/apiUtils/object/storeObject.js

claude · 2026-03-17T16:01:31Z

Arsenal dependency is pinned to a branch instead of a tag. Must be pinned to a release tag before merging.
- Change arsenal in package.json to point to a tagged version.
Verify ObjectMDChecksum constructor argument names match the field names used in storeObject.js checkHashMatchMD5 (algorithm, value, type). The unit tests assert checksumAlgorithm, checksumValue, checksumType on the stored metadata. If ObjectMDChecksum maps these internally it is fine, but worth confirming.
In storeObject.js dataStore, the onStreamError handler (line 91) and doValidate via once('finish', ...) (line 125) can both fire if the stream emits both error and finish. jsutil.once(cb) prevents a double callback, but both paths would call batchDelete independently. Consider removing the opposing listener when one fires to avoid a redundant delete call.

Review by Claude Code

package.json

lib/api/apiUtils/object/storeObject.js

claude · 2026-03-17T16:50:33Z

Review summary: (1) Arsenal dependency pinned to a branch instead of a tag - must update to a released tag. (2) checkHashMatchMD5 JSDoc missing the new checksumStream parameter. (3) dataStore callback returns a 4th checksum arg but objectPutPart and routeBackbeat callers ignore it. -- Review by Claude Code

package.json

claude · 2026-03-17T17:26:40Z

Arsenal dependency is pinned to a branch instead of a tag, which makes builds non-reproducible. Pin to a released tag before merging.

Review by Claude Code

package.json

claude · 2026-03-17T19:07:51Z

Arsenal dependency is pinned to a feature branch instead of a release tag. Pin to a tag before merging.

Review by Claude Code

package.json

claude · 2026-03-17T19:41:05Z

Arsenal is pinned to a feature branch instead of a release tag. This must be updated to a tagged version before merging to ensure reproducible builds.

Review by Claude Code

package.json

lib/routes/routeBackbeat.js

claude · 2026-03-18T12:01:21Z

Arsenal dependency is pinned to a branch instead of a tag. Must be updated to a released tag before merge.
Misleading comment placement in routeBackbeat.js (two places): the checksum 4th-arg comment sits between dataStore positional args. Move it above the callback.

Review by Claude Code

package.json

lib/api/apiUtils/object/createAndStoreObject.js

claude · 2026-03-18T12:12:49Z

Arsenal dependency is pinned to a feature branch instead of a release tag. Must be updated to a tagged version before merging.
- Pin to the release tag that includes the ObjectMDChecksum changes.
zeroSizeBodyChecksumCheck mixes callbacks with Promises, creating a double-callback risk: if callback throws inside .then(), the .catch() will invoke callback(err) a second time.
- Use process.nextTick inside the .then() and .catch() to break out of the promise chain, or convert to async/await with try/catch.

Review by Claude Code

package.json

claude · 2026-03-18T12:21:25Z

Arsenal dependency is pinned to a branch instead of a tag. This must be updated to a release tag before merging for reproducible builds.

Review by Claude Code

lib/api/apiUtils/object/createAndStoreObject.js

…bject

package.json

claude · 2026-03-20T15:47:29Z

Arsenal dependency is pinned to a feature branch instead of a release tag. Must be updated to a tag before merge.

Review by Claude Code

claude · 2026-03-20T15:52:43Z

LGTM

Review by Claude Code

BourgoisMickael · 2026-03-23T11:17:36Z

tests/functional/raw-node/test/routes/routeMetadata.js

+        await s3.send(new DeleteBucketCommand({ Bucket: bucket }));
+    });
+
+    it('stores sha256 checksum in metadata when x-amz-checksum-sha256 is provided', done => {


prefer starting with should

fredmnl · 2026-03-25T09:48:27Z

tests/functional/raw-node/test/checksumPutObjectUploadPart.js

+                        assert.strictEqual(res.headers['x-amz-checksum-crc64nvme'], crc64nvmeOfTestContent2,
+                            `expected x-amz-checksum-crc64nvme: ${crc64nvmeOfTestContent2}`);
+                    }
+                    done();


Should we fail this test if the checksum header is absent?

Same for all tests below

fredmnl · 2026-03-25T09:57:08Z

tests/functional/raw-node/test/checksumPutObjectUploadPart.js

+        crc32c: 'AAAAAA==',
+        sha1: '2jmj7l5rSw0yVb/vlWAYkK/YBwk=',
+        sha256: '47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=',
+        crc64nvme: null, // filled in before hook (async)


nit: Not sure why we're not writing this one down as well

leif-scality changed the base branch from development/9.4 to improvement/CLDSRV-863-checksums-put-object-part March 17, 2026 15:51

claude bot reviewed Mar 17, 2026

View reviewed changes

package.json Outdated Show resolved Hide resolved

claude bot reviewed Mar 17, 2026

View reviewed changes

lib/api/apiUtils/object/storeObject.js Show resolved Hide resolved

leif-scality force-pushed the improvement/CLDSRV-872-store-checksum-in-object-metadata branch from 5d6c434 to 1b5a063 Compare March 17, 2026 16:48

claude bot reviewed Mar 17, 2026

View reviewed changes

package.json Outdated Show resolved Hide resolved

claude bot reviewed Mar 17, 2026

View reviewed changes

lib/api/apiUtils/object/storeObject.js Show resolved Hide resolved

claude bot reviewed Mar 17, 2026

View reviewed changes

lib/api/apiUtils/object/storeObject.js Show resolved Hide resolved

leif-scality force-pushed the improvement/CLDSRV-872-store-checksum-in-object-metadata branch from 1b5a063 to b80b2ad Compare March 17, 2026 17:24

claude bot reviewed Mar 17, 2026

View reviewed changes

package.json Outdated Show resolved Hide resolved

claude bot reviewed Mar 17, 2026

View reviewed changes

package.json Outdated Show resolved Hide resolved

claude bot reviewed Mar 17, 2026

View reviewed changes

package.json Outdated Show resolved Hide resolved

claude bot reviewed Mar 18, 2026

View reviewed changes

package.json Outdated Show resolved Hide resolved

claude bot reviewed Mar 18, 2026

View reviewed changes

lib/routes/routeBackbeat.js Outdated Show resolved Hide resolved

claude bot reviewed Mar 18, 2026

View reviewed changes

package.json Outdated Show resolved Hide resolved

claude bot reviewed Mar 18, 2026

View reviewed changes

lib/api/apiUtils/object/createAndStoreObject.js Outdated Show resolved Hide resolved

claude bot reviewed Mar 18, 2026

View reviewed changes

package.json Outdated Show resolved Hide resolved

BourgoisMickael reviewed Mar 19, 2026

View reviewed changes

lib/api/apiUtils/object/createAndStoreObject.js Outdated Show resolved Hide resolved

leif-scality force-pushed the improvement/CLDSRV-863-checksums-put-object-part branch from a284c9f to dade07b Compare March 20, 2026 09:54

CLDSRV-863: handle trailer headers and add new trailer errors

9276864

CLDSRV-872: store checksum in metadata for objects uploaded with PutO…

1dfbae8

…bject

leif-scality force-pushed the improvement/CLDSRV-872-store-checksum-in-object-metadata branch from 180d940 to 7d130cc Compare March 20, 2026 15:45

claude bot reviewed Mar 20, 2026

View reviewed changes

package.json Outdated Show resolved Hide resolved

leif-scality added 8 commits March 20, 2026 16:47

CLDSRV-872: return stored checksum in PutObject response

16f2b1e

CLDSRV-872: handle empty object checksum

d1ebaee

CLDSRV-872: fix failing test, checksum lost when restoring object

3ffba38

CLDSRV-872: test checksums are respected in backbeat routes

6521749

CLDSRV-872: add missing jsdoc

c524ec8

fixup! CLDSRV-863: PutObject/UploadPart checksum service tests

9ff842a

fixup! CLDSRV-863: PutObject/UploadPart checksum service tests

9912f20

CLDSRV-863: bump arsenal to 8.3.7

8e51413

leif-scality force-pushed the improvement/CLDSRV-872-store-checksum-in-object-metadata branch from 7d130cc to 8e51413 Compare March 20, 2026 15:49

leif-scality marked this pull request as ready for review March 20, 2026 16:32

leif-scality requested a review from BourgoisMickael March 20, 2026 20:53

BourgoisMickael reviewed Mar 23, 2026

View reviewed changes

BourgoisMickael approved these changes Mar 23, 2026

View reviewed changes

fredmnl self-requested a review March 23, 2026 17:28

fredmnl approved these changes Mar 25, 2026

View reviewed changes

Conversation

leif-scality commented Mar 17, 2026

Uh oh!

bert-e commented Mar 17, 2026

Hello leif-scality,

Uh oh!

bert-e commented Mar 17, 2026

Incorrect fix version

Uh oh!

codecov bot commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ 2 Tests Failed:

Uh oh!

Uh oh!

Uh oh!

claude bot commented Mar 17, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

claude bot commented Mar 17, 2026

Uh oh!

Uh oh!

claude bot commented Mar 17, 2026

Uh oh!

Uh oh!

claude bot commented Mar 17, 2026

Uh oh!

Uh oh!

claude bot commented Mar 17, 2026

Uh oh!

Uh oh!

Uh oh!

claude bot commented Mar 18, 2026

Uh oh!

Uh oh!

Uh oh!

claude bot commented Mar 18, 2026

Uh oh!

Uh oh!

claude bot commented Mar 18, 2026

Uh oh!

Uh oh!

Uh oh!

claude bot commented Mar 20, 2026

Uh oh!

claude bot commented Mar 20, 2026

Uh oh!

BourgoisMickael Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

fredmnl Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

fredmnl Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

fredmnl Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov bot commented Mar 17, 2026 •

edited

Loading