Add thv skill sync: restore project skills from the lock file#5895
Conversation
784a7ce to
9c2d5f5
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #5895 +/- ##
==========================================
- Coverage 71.93% 71.88% -0.06%
==========================================
Files 711 713 +2
Lines 73029 73228 +199
==========================================
+ Hits 52534 52638 +104
- Misses 16726 16822 +96
+ Partials 3769 3768 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
7824f3f to
1514a9d
Compare
9c2d5f5 to
06a66f8
Compare
1514a9d to
3dd34f4
Compare
06a66f8 to
42a03de
Compare
42a03de to
4604083
Compare
3b4e60f to
f8fe482
Compare
773d3ff to
78b4c7d
Compare
JAORMX
left a comment
There was a problem hiding this comment.
Panel review (spec / standards / security / library-reuse / duplication / golang-conventions / UX). The sync core is correct — content re-hash verifies every client dir, --check is read-only, --adopt/--prune behave, and restore installs exactly the pinned resolvedReference@digest. A few real findings; also two items that look broken here but are completed by #5897 (noted so they aren't double-counted). Commenting rather than approving.
Real findings:
-
--clients "all"is passed through literally —skill_sync.go:81help promises"all"expands to every detected client, butparseSkillInstallClients(skill_install.go:67) has no such handling, soSyncreceives["all"]verbatim. Implement the expansion or fix the help text. -
Git restore doesn't verify the digest is a pinned commit (Medium, security) —
pinGitReference(pin.go:67) buildsgit://…@<entry.Digest>and passes it toCloneConfigForRef, which only treats it as a commit when it's 40-char hex. A tampered/hand-edited lock entry withdigest: main(or a tag) silently pulls the branch tip instead of the pinned content — the restore invariant is violated. Reject non-40-char-hex digests inpinGitReference(mirror whatpinOCIReferencegets free from a digest ref). -
pinGitReferenceassumes anhttps?://prefix —pin.go:69TrimPrefix(TrimPrefix(url, "https://"), "http://")silently no-ops on any other form, producing a corrupt pinned reference rather than an error. Return an error when neither prefix matches.
Stacking artifacts (fixed by #5897 — no action here):
sync --checkalways exits 0 on drift — resolved by #5897's typed exit codes (exit 2 on check failure).isImmutableSource/isFullCommitHashlook like dead code — #5896 is their caller.
UX (should-fix): --check output is indistinguishable from a real sync (no "no changes made" banner); successfully-adopted skills still appear under "Never managed (use --adopt to record)"; "the project matches its lock file" prints even when no lock file exists. Also the TOOLHIVE_SKILLS_LOCK_ENABLED gate is undocumented in --help/docs — a user following the help hits a dead-end 403.
Security (verified clean): OCI restore is content-addressed, projectRoot is validated (absolute/canonical/git-rooted, CWE-22), prune delegates to the scoped Uninstall, --adopt is explicit opt-in, and POST /skills/sync sits behind auth middleware.
f8fe482 to
8bca3c7
Compare
78b4c7d to
bad7ade
Compare
bad7ade to
187a35d
Compare
8bca3c7 to
1712886
Compare
RFC THV-0080 needs a way to restore a project's pinned skill set on any machine (a fresh clone, CI, a teammate's laptop) and to verify on-disk content still matches the lock file without installing anything. This adds the full sync vertical slice: service logic, HTTP endpoint, client, and CLI command. - Sync reconciles installed skills against toolhive.lock.yaml: missing/drifted entries reinstall at the pinned reference (never re-resolving source, see buildPinnedReference), --check reports drift read-only, --adopt/--prune handle unlocked installs - Fixes a real bug this surfaced: reinstalling at an unchanged pinned digest hit both the OCI and git install paths' "same digest means content is already correct" fast path, silently skipping re-extraction — exactly the drift sync exists to repair. A new internal SyncRestore option bypasses that fast path. - POST /api/v1beta/skills/sync via a narrow skillSyncer interface (pkg/api/v1), so this PR doesn't need to ship an Upgrade stub - `thv skill sync` CLI (--check/--adopt/--prune/--clients/ --project-root, auto-detected via .git when omitted) Part of the production stack for #5715 (RFC THV-0080). Stack: 4/6.
reinstallPinned reinstalls at a pinned reference derived from the lock entry, and that pinned string was flowing straight into ResolvedReference via the install path — a drift repair overwrote it with an internal restore form instead of preserving what Source had actually resolved to, which later corrupts upgrade's ref-change detection. Add a LockResolvedReference override on InstallOptions, mirroring the existing LockSource one.
Picks up the AlreadyCurrent JSON tag rename and Drifted doc comment fix from the PR2 review pass.
The panel review's blocker: drift was only recorded for entries with an install record, so a lock entry with none — every entry on a fresh clone or CI runner, since SQLite state is per-machine — landed in no result bucket and sync --check green-lit a checkout missing all of its locked skills. Add a Missing bucket, recorded before the check return. Restores had three related correctness gaps: restoring a parent re-materialized its toolhive.requires from mutable source strings (silently upgrading and re-pinning a dependency whose upstream tag moved), a restore promoted non-explicit dependencies to explicit (permanently exempting them from cascade removal), and verification hashed only the first client's directory (tampering with any other client's copy was invisible, install-order-dependently). Skip requires recursion on SyncRestore, preserve Explicit verbatim, and verify every client directory. Also classify lock-write failures via a sentinel instead of mapping every HTTP 500 to lock-write-failed, and fix the E2E decode struct reading AlreadyCurrent from a stale JSON key — a silently-passing assertion.
1712886 to
d8f23e4
Compare
Summary
syncvertical slice in one PR since the CLI command can't exist without its API endpoint.Syncreconciles installed skills againsttoolhive.lock.yaml: missing/drifted entries reinstall at the pinned reference (never re-resolvingsource— seebuildPinnedReference),--checkreports drift read-only,--adoptrecords lock entries for unmanaged installs,--pruneremoves installs no longer in the lock.syncexists to repair (a tampered file doesn't change the pinned commit/digest). A new internalSyncRestoreoption onInstallOptionsbypasses that fast path in bothinstall_extraction.goandinstall_git.go.POST /api/v1beta/skills/sync, wired through a narrowskillSyncerinterface inpkg/api/v1so this PR doesn't need to ship anUpgradestub (widens toskills.SkillLockServicein PR5).thv skill syncCLI:--check,--adopt,--prune,--clients,--project-root(auto-detected by walking up for.gitwhen omitted).Part of the production stack for #5715 (RFC THV-0080). Stack: 4/6 — lockfile → lock-service → install-hooks → sync → upgrade → cli-exitcodes. Based on #5894 (PR3).
Type of change
Test plan
task test)task test-e2e, scoped:ginkgo --label-filter=skills-lock)task lint-fix)Table-driven drift-matrix tests cover up-to-date / drifted-and-reinstalled / missing-and-restored / check-mode-no-write / adopt / prune / feature-disabled-403; golden tests for
buildPinnedReference(OCI digest-pin, git commit-pin with and without a subdir path) andisImmutableSource; API handler tests including the 501 "not supported" path when the underlying service lacksSync. New E2E spec: install → delete files on disk →sync --checkreports drift without touching anything →syncrestores the files.Does this introduce a user-facing change?
Not yet — same
TOOLHIVE_SKILLS_LOCK_ENABLEDgate as the rest of the stack (see PR3). Once the full stack lands:thv skill syncrestores a project's skills to match its committed lock file.Special notes for reviewers
SyncRestorefix (install_extraction.go,install_git.go) is the part I'd like the most scrutiny on — it's a small diff but changes behavior on the existing install fast paths, gated entirely behind an internal-only option onlySyncsets.pkg/skills/client.ClientgainsSyncbut does not yet assertskills.SkillLockServiceconformance — that lands in PR5 onceUpgradeexists too.🤖 Generated with Claude Code