Skip to content

feat: add cron cleanup job for expired shares, trashed snippets, stale logs, and backup files - #165

Merged
SudiptaPaul-31 merged 2 commits into
SudiptaPaul-31:mainfrom
JemimahEkong:feature/cleanup-jobs
Aug 30, 2026
Merged

feat: add cron cleanup job for expired shares, trashed snippets, stale logs, and backup files#165
SudiptaPaul-31 merged 2 commits into
SudiptaPaul-31:mainfrom
JemimahEkong:feature/cleanup-jobs

Conversation

@JemimahEkong

Copy link
Copy Markdown
Contributor

feat: scheduled cleanup system for expired data and stale records

What

Introduces a scheduled cleanup system that runs daily at 03:17 UTC via Vercel Cron. It purges expired share links, soft-deleted snippets past retention, stale activity/analytics/audit logs, expired auth sessions, and old encrypted backup files.

Why

Without automated cleanup, expired shares, trashed snippets, and logs accumulate indefinitely — growing the database, leaking stale data past grace periods, and accumulating backup files on disk.

Changes

Core feature

  • app/api/cron/cleanup/route.ts — GET endpoint protected by CRON_SECRET Bearer auth
  • lib/cleanup.service.ts — Orchestrates 5 independent tasks, continues on individual failures, returns a structured summary with per-task counts, durations, and errors
  • lib/cleanup.repository.ts — Batched DELETE queries (500 rows/batch, max 20 batches) for snippet_shares, snippets, auth_sessions, login_nonces, activity_logs, snippet_analytics, and audits
  • lib/cleanup.service.test.ts — Tests covering service logic, config parsing, backup file pruning, error resilience, and batch behavior
  • vercel.json — Scheduled cron route

Retention defaults (configurable via env vars)

Cleanup Task Env Var Default
Expired/revoked shares CLEANUP_SHARE_GRACE_DAYS 7 days
Soft-deleted snippets TRASH_RETENTION_DAYS 30 days
Encrypted backup files BACKUP_RETENTION_DAYS 30 days
Activity/analytics/audit logs LOG_RETENTION_DAYS 90 days
Expired auth sessions/nonces Always purged

Fixes and improvements

  • lib/activity-logger.ts — Lazy-init DB client to prevent crash when DATABASE_URL is unset; remove duplicate ResourceType export
  • lib/reputation.ts — Lazy-init DB client instead of failing at module load
  • lib/utils.ts — Add eslint-disable for intentional control-char regex
  • app/api/snippets/[id]/transfer-ownership/route.tsconst to let for reassignable variable

Test infrastructure

  • Switch jest default env to node (Node 22 has native fetch), add jest.setup.js for TextDecoder/TextEncoder polyfill
  • Fix lib/snippet.service.test.ts: correct IPFSService mock path, add missing findById mock for update-not-found case
  • Add @jest-environment jsdom pragma to validiation/frontend-signature.test.tsx

Design notes

  • Batched deletes (500 rows/batch, capped at 20 batches per task) avoid long-held locks or timeout risk on large tables during a single cron invocation.
  • Independent task failure isolation: each of the 5 cleanup tasks runs independently and failures in one don't block the others — the endpoint returns a structured summary with per-task counts, durations, and errors so partial failures are visible and debuggable rather than silent.
  • Auth: the cron endpoint is protected by CRON_SECRET Bearer auth so it can't be triggered by unauthenticated requests.
  • Safe deletion: all deletes are scoped strictly by expiration/retention timestamps computed from the configurable env vars above — active/non-expired records are never touched.

Testing

  • lib/cleanup.service.test.ts covers service orchestration logic, config parsing (env var overrides + defaults), backup file pruning, error resilience when an individual task fails, and batch-size/batch-count behavior.
  • Full test suite passes after the jest environment and mock-path fixes described above.

Acceptance criteria

  • Automated scheduling — cron runs daily at 03:17 UTC with no manual intervention
  • Safe deletion — all deletes scoped to expired/stale records only, verified by tests
  • Logging & monitoring — structured summary with per-task counts, durations, and errors returned on every run
  • Configurable intervals/thresholds — retention windows configurable via env vars per task
  • Performance gains — no before/after measurement included in this PR; recommend monitoring DB size/query load post-deploy to confirm

Related

Closes #131

…e logs, and backup files

- Add CleanupService and CleanupRepository with batched deletes
- Add /api/cron/cleanup route with Bearer auth
- Schedule cleanup daily at 03:17 via vercel.json cron
- Lazy-init DB clients in activity-logger and reputation to avoid crashes when DATABASE_URL is missing
- Fix duplicate ResourceType export in activity-logger
- Add eslint-disable for intentional control-char regex
- Fix frontend-signature test env to jsdom
- Update jest config: node env, TextDecoder polyfill in setup
- Add missing test mocks for ipfs service and snippet update flow
- Add undici and testing-library dev dependencies
@vercel

vercel Bot commented Aug 25, 2026

Copy link
Copy Markdown

@JemimahEkong is attempting to deploy a commit to the Sudipta 's projects Team on Vercel.

A member of the Team first needs to authorize it.

@drips-wave

drips-wave Bot commented Aug 25, 2026

Copy link
Copy Markdown

@JemimahEkong Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@SudiptaPaul-31

Copy link
Copy Markdown
Owner

@JemimahEkong resolve conflicts

@vercel

vercel Bot commented Aug 29, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
codely Error Error Aug 29, 2026 4:35pm
v0-code-snippets-platform-24 Error Error Aug 29, 2026 4:35pm

@vercel

vercel Bot commented Aug 29, 2026

Copy link
Copy Markdown

Deployment failed for project v0-code-snippets-platform-24 with the following error:

Invalid vercel.json file provided

@SudiptaPaul-31
SudiptaPaul-31 merged commit f14295f into SudiptaPaul-31:main Aug 30, 2026
1 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scheduled Cleanup Jobs

2 participants