feat: add on-demand signature generation

[ferhatelmas]

What kind of change does this PR introduce?

feat

What is the current behavior?

We don't have signatures of the objects.
Generating signatures in the hot path is too costly, and hash and multipart support makes it infeasible on upload in every case.

What is the new behavior?

Register two events, multi signatures (tenant, bucket, batch objects) and single signature (specific version of an object).
Add an admin on-demand handler to schedule coordinator job (multi) which reschedules itself as needed and schedules single object jobs.

Additional context

Single object can be triggered upon successful upload upon byte change as a follow up. Not connected it yet to see the real performance and bottlenecks.
Partial index will introduce a small overhead for maintenance.

[Copilot]

Pull request overview

Adds on-demand object SHA-256 signature generation for storage objects through new queue events, database accessors, a tenant migration, and an admin scheduling endpoint.

Changes:

• Adds coordinator and per-object queue events for object signature generation.
• Adds DB methods and migration support for storage.objects.signature.
• Adds an admin API route to schedule signature generation jobs.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
src/storage/events/workers.ts	Registers the new signature generation workers.
src/storage/events/objects/generate-object-signature.ts	Implements coordinator and per-object signature generation jobs.
src/storage/events/objects/generate-object-signature.test.ts	Adds unit coverage for signature generation events.
src/storage/events/index.ts	Exports the new events.
src/storage/database/knex.ts	Adds object listing and signature update DB methods.
src/storage/database/knex.test.ts	Tests the new DB query builders.
src/storage/database/adapter.ts	Extends the database interface for signature operations.
src/internal/monitoring/logger.ts	Allows event metadata and error event fields.
src/internal/database/migrations/types.ts	Registers migration 61.
src/http/routes/admin/signature-generation.ts	Adds admin endpoint to enqueue signature generation.
src/http/routes/admin/signature-generation.test.ts	Tests admin endpoint auth, validation, and enqueueing.
src/http/routes/admin/index.ts	Exports the new admin route.
src/admin-app.ts	Registers the new admin route.
migrations/tenant/0061-add-objects-signature.sql	Adds the signature column and length constraint.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[coveralls]

Coverage Report for CI Build 26082139452

Coverage increased (+0.4%) to 75.443%

Details

• Coverage increased (+0.4%) from the base build.
• Patch coverage: 13 uncovered changes across 2 files (165 of 178 lines covered, 92.7%).
• No coverage regressions found.

Uncovered Changes

File	Changed	Covered	%
src/storage/events/objects/generate-object-signature.ts	132	121	91.67%
src/storage/events/workers.ts	2	0	0.0%

Coverage Regressions

No coverage regressions found.

---

Coverage Stats

Relevant Lines:	10484
Covered Lines:	8339
Line Coverage:	79.54%
Relevant Branches:	6049
Covered Branches:	4134
Branch Coverage:	68.34%
Branches in Coverage %:	Yes
Coverage Strength:	407.53 hits per line

---

💛 - Coveralls

[Copilot]

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 6 comments.

Comments suppressed due to low confidence (4)

src/test/tus.test.ts:413

• This loosens a regression assertion that previously verified the complete persisted object shape. Since the new signature: null field is already included in the expected object, keeping exact equality would still cover the schema change while catching unexpected extra fields.

    expect(dbAsset).toEqual(
      expect.objectContaining({

src/test/tus.test.ts:571

• This loosens a regression assertion that previously verified the complete persisted object shape. Since the new signature: null field is already included in the expected object, keeping exact equality would still cover the schema change while catching unexpected extra fields.

    expect(dbAsset).toEqual(
      expect.objectContaining({

src/test/tus.test.ts:844

• This loosens a regression assertion that previously verified the complete persisted object shape. Since the new signature: null field is already included in the expected object, keeping exact equality would still cover the schema change while catching unexpected extra fields.

      expect(dbAsset).toEqual(
        expect.objectContaining({

src/test/tus.test.ts:916

• This loosens a regression assertion that previously verified the complete persisted object shape. Since the new signature: null field is already included in the expected object, keeping exact equality would still cover the schema change while catching unexpected extra fields.

      expect(dbAsset).toEqual(
        expect.objectContaining({

[Copilot]

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 3 comments.

[Copilot]

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

src/storage/events/objects/generate-object-signature.ts:200

• The job reads the backend object before checking that the database row still points at this version. If the object is overwritten/deleted after the coordinator enqueues the job, the old backend version may already be removed, causing getObject to retry/dead-letter a stale job instead of treating it as skipped; checking the row/version before downloading would avoid those false failures.

      const response = await storage.backend.getObject(
        storageS3Bucket,
        objectPath,
        job.data.version
      )
      const sha256 = await digestObjectBody(response.body)

[Copilot]

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 5 comments.

[Copilot]

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 1 comment.