Skip to content

Migrate package manager from yarn to npm#6216

Open
pawelangelow wants to merge 25 commits into
mainfrom
migrate-to-npm
Open

Migrate package manager from yarn to npm#6216
pawelangelow wants to merge 25 commits into
mainfrom
migrate-to-npm

Conversation

@pawelangelow

@pawelangelow pawelangelow commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

What

Migrates the entire repo from Yarn to npm — all package directories plus CI, Docker,
build scripts, docker-compose, and docs. Behaviour-preserving: no dependency versions
change.

Dependencies (all 10 package dirs: root, redisinsight, redisinsight/api, and the
7 ui/src/packages/* plugins):

  • resolutions → npm overrides (glob → nested syntax); resolved 4 EOVERRIDE
    direct-dependency conflicts (semver, styled-components, @types/react[-dom])
  • package-lock.json seeded from yarn.lock (via synp) then finalized with
    npm install, so npm reproduces yarn's exact tree — verified zero new versions
    vs yarn.lock in every dir, and npm ci --dry-run passes
  • .npmrc with legacy-peer-deps=true per dir (mirrors yarn's lenient peer resolution;
    required by @elastic/eui@34.6.0 vs React 18)
  • sass alias pinned to yarn's version; engines drop yarn / require npm >=8.3.0;
    removed yarn-deduplicate/skip-postinstall/postinstall-postinstall; simplified
    postinstall; deleted yarn.lock and .yarnrc

Scripts / CI / Docker:

  • package.json scripts: yarn --cwd X Ynpm run Y --prefix X
  • workflows + composite actions: yarn install --frozen-lockfilenpm ci
  • Dockerfile, build.sh, build_modules.sh, build-statics.{sh,cmd}, test
    Dockerfiles, and integration-test docker-compose.yml files → npm
  • docs, READMEs, and .ai/skills/* command examples → npm

Known behaviour change: yarn autoclean has no npm equivalent, so it was dropped —
release artifacts / Docker image are slightly larger. Flagged in-file as a node-prune
follow-up.

Testing

Local (all pass):

  • npm ci in root, redisinsight/api, redisinsight → app installs; postinstall
    (patch-package + vite optimize) runs clean
  • npm run type-check, npm run lint, npm test, npm run test:api
  • npm run build:statics (installs + builds all 7 plugins), npm run dev:desktop

CI: verify the workflows on this PR (backend, frontend, e2e, type-check, builds) — the
CI/Docker conversion can only be validated on a real run.


Note

High Risk
This touches every install, CI job, Docker image, and release build path; a missed yarn reference or lockfile drift will break pipelines, and dropping yarn autoclean changes artifact size.

Overview
This PR replaces Yarn with npm across the repo while aiming to keep the same dependency tree and app behavior.

Tooling and lockfiles: Yarn resolutions become npm overrides (with documented pins for semver, @types/react*, etc. in docs/npm-migration-decisions.md). package-lock.json files replace yarn.lock; .yarnrc is removed. Root and per-package .npmrc set legacy-peer-deps=true to mirror Yarn’s peer handling. Scripts move from yarn / yarn --cwd to npm run / npm run … --prefix.

CI, Docker, and release: Composite actions and workflows use npm ci (and npm ci --ignore-scripts when postinstall must be skipped). Cache keys hash package-lock.json instead of yarn.lock. The post-install yarn-deduplicate lockfile check is removed. Dockerfile, .github/build/*, and platform package pipelines invoke npm; yarn autoclean is dropped (noted as larger release/Docker artifacts).

Docs and developer UX: AGENTS.md, CONTRIBUTING.md, .ai/skills/*, VS Code launch config, and plugin deploy templates are updated to npm commands. .gitignore stops ignoring .npmrc so peer-dep settings are committed. The tests workflow infra path filter watches package-lock.json and .npmrc instead of Yarn files.

Reviewed by Cursor Bugbot for commit 6c033ce. Bugbot is set up for automated code reviews on this repo. Configure here.

pawelangelow and others added 22 commits July 16, 2026 14:34
First step of the yarn -> npm migration: translate the `resolutions`
blocks to npm `overrides` in the four core package.json files (root,
redisinsight, redisinsight/api, redisinsight/ui).

Syntax-only translation; no dependency versions change, so npm forces
the same versions yarn did:
- `**/pkg` and bare `pkg` -> top-level override (applies at every depth)
- `parent/pkg` -> nested `{ "parent": { "pkg": ... } }`
- deep paths preserved (react-router-dom > react-router > path-to-regexp)
- `@elastic/eui/**/prismjs` -> `{ "@elastic/eui": { "prismjs": ... } }`
- api: duplicate `**/form-data` + `form-data` merged into one entry

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Continue the yarn -> npm migration: rewrite package.json scripts to run
via npm instead of yarn (root, api, desktop, ui). Behaviour unchanged.

- `yarn --cwd <dir> <script>` -> `npm run <script> --prefix <dir>`
- same-package `yarn <script>` / `yarn run <script>` -> `npm run <script>`
- args after the script name now pass through `--`, e.g.
  `yarn --cwd ui build --emptyOutDir`
    -> `npm run build --prefix redisinsight/ui -- --emptyOutDir`
  `yarn typeorm migration:run` -> `npm run typeorm -- migration:run`
- updated the tscheck hint strings handed to ts-error-check.ts

The root `postinstall` still references yarn-deduplicate; it is cleaned
up in a follow-up commit together with the yarn-only devDependencies.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remove the yarn-only pieces now that scripts run through npm:

- postinstall: drop the `&& skip-postinstall || yarn-deduplicate
  yarn.lock` tail, leaving `patch-package && vite optimize`
- devDependencies: remove `yarn-deduplicate`, `skip-postinstall`,
  `postinstall-postinstall` (all only existed to support the yarn
  postinstall/dedupe flow)
- engines: drop the `yarn` requirement and raise `npm` to `>=8.3.0`
  (the minimum that supports the `overrides` field)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Pin the `sass` alias to the exact version yarn.lock resolves
(`npm:sass-embedded@1.75.0`). Without a version, npm re-resolves the
alias to the latest sass-embedded on install (observed drift to
1.100.0), which is a silent behaviour change and also drops sass's
transitive `buffer-builder`.

Deliberately does NOT add a separate `sass-embedded` dependency: yarn
only had the `sass` alias (node_modules/sass), so Vite resolves it via
the legacy Sass API. Adding sass-embedded directly would switch Vite to
the modern compile API — a behaviour change we are avoiding to keep the
migration faithful. 1.75.0 predates the @import/mixed-decls/legacy-js-api
deprecation warnings, so no warnings surface at this version.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add `.npmrc` with `legacy-peer-deps=true` at the root, redisinsight/ and
redisinsight/api/ (the core install dirs). This reproduces yarn's lenient
peer-dependency resolution; @elastic/eui@34.6.0 declares legacy peers
(e.g. @types/react@^16) that otherwise conflict with React 18 under npm.

- .gitignore: stop ignoring `.npmrc` so the files are tracked
- remove `.yarnrc` (yarn-only config)

.npmrc for the ui/src/packages/* plugin dirs is deferred pending the
plugin install-scope decision.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Seed the root lockfile from yarn.lock (synp) and reconcile with
`npm install --package-lock-only --ignore-scripts --legacy-peer-deps`,
so npm reproduces yarn's exact tree instead of resolving fresh.

Resolve the override/direct-dependency conflicts npm surfaced
(EOVERRIDE), documented in docs/npm-migration-decisions.md:
- semver: pin override + direct dep to exact 7.7.2 (yarn collapsed all
  semver to 7.7.2; dropping the override drifted to 5 versions)
- styled-components: drop redundant override (direct ^5.0.0 already
  caps <6; both resolve 5.3.11)
- @types/react, @types/react-dom: pin direct devDeps to exact 18.2.1 to
  match the 18.2.1 override

Verified: zero version drift and zero unexpected removals vs yarn.lock
(only the intentionally removed yarn-only devDeps and npm: alias
false-positives differ). yarn.lock is kept until the CI cutover.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Seed from api yarn.lock (synp) and reconcile with
`npm install --package-lock-only --ignore-scripts --legacy-peer-deps`.

No package.json changes were needed: the api overrides had no
direct-dependency conflicts, and seeding preserved yarn's versions
(e.g. semver stayed at 7.5.4 rather than floating off the ^7.5.2 floor).

Verified against yarn.lock:
- npm introduced ZERO new versions (every difference is npm collapsing
  redundant duplicate versions yarn kept; npm's set is a subset of
  yarn's, all within range)
- the cpu-features file: stub is materialised
- no unexpected removals (only dedup byproducts and npm: alias
  false-positives)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the missing lockfile for the redisinsight/ desktop workspace (the
prototype deleted its yarn.lock without generating a package-lock).

Method note: synp requires an existing node_modules to seed, and
redisinsight/ has none, so the lockfile was produced by
`npm install --package-lock-only --ignore-scripts --legacy-peer-deps`
and its fidelity verified against yarn.lock by diff instead:
- NEW versions introduced by npm: NONE
- semver 7.5.4, better-sqlite3 12.10.1, keytar 7.9.0, tunnel-ssh 5.1.2,
  ssh2 1.16.0 all match yarn exactly
- cpu-features file: stub materialised

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`npm install --package-lock-only` produced lockfiles missing the optional
platform-specific native binaries (@esbuild/*, @rollup/*, @swc/*,
@sentry/cli-*, @msgpackr-extract/*) for non-host OSes, so `npm ci`
rejected them (EUSAGE "Missing ... from lock file"). CI runs npm ci, so
this would have failed there too.

Finalised by running a real `npm install`, which records every platform
variant (with os/cpu) — the same set yarn.lock had:
- root: +97 platform binaries
- api: +25 platform binaries; npm also deduped 9 more @babel/browserslist
  duplicates than --package-lock-only did

Verified: `npm ci --dry-run` now passes in root, api and redisinsight,
and there are no new versions vs yarn (only alias naming artifacts such
as sass -> sass-embedded). redisinsight/package-lock.json was unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Convert the yarn `resolutions` blocks to npm `overrides` in the 6
Workbench plugin packages under redisinsight/ui/src/packages/
(clients-list, geodata, redisearch, redisgraph, redistimeseries-app,
ri-explain). Same glob->nested translation as the core packages;
syntax-only, no version changes. The `@elastic/eui/**/prismjs` entries
become nested `@elastic/eui: { prismjs }` overrides (eui is a direct dep,
but the override targets its child, so no EOVERRIDE).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add package-lock.json and .npmrc (legacy-peer-deps=true) for the plugin
aggregate and all 6 sub-plugins under redisinsight/ui/src/packages/.

These dirs have no node_modules, so synp can't seed them; lockfiles were
produced by a real `npm install` (aggregate first, so its node_modules/
vite satisfies the plugins' "vite": "file:../node_modules/vite" link) and
verified against each yarn.lock: zero new versions (the only diff is the
redisinsight-plugin-sdk file: dep, a naming artifact). `.npmrc` is
required because @elastic/eui@34.6.0's legacy peers otherwise ERESOLVE
against React 18. `npm run build` builds all plugins cleanly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the yarn plugin install/build calls with npm:
- `yarn --cwd <plugin>` -> `npm install --prefix <plugin>` (aggregate
  first, then each sub-plugin, preserving the order the file: vite link
  needs)
- `yarn --cwd "${PACKAGES_DIR}" build` -> `npm run build --prefix ...`

Verified end to end: all 6 plugins install and build via npm.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Delete all 10 yarn.lock files now that every dir has a verified
package-lock.json (root, redisinsight, api, and the 7 plugin packages).
They remain recoverable from git history until the CI/Docker cutover is
confirmed green.

Fidelity verified before removal: the no-new-versions gate passes in all
10 dirs (npm introduced no version yarn.lock did not already have;
differences are npm deduping duplicates plus file:/alias naming
artifacts), npm ci --dry-run passes, patch-package applies, and the app
+ plugins build/run on npm.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- install-deps action: `yarn install --frozen-lockfile` -> `npm ci`
  (honours the skip-postinstall input via `npm ci --ignore-scripts`)
- install-all-build-libs: cache key + hashFiles use package-lock.json;
  generate:api-client runs via `npm run ... --prefix`
- workflows (lint, licenses-check, i18n, type-check, tests-frontend,
  tests-backend, tests-e2e-playwright-chromium, pipeline-build-*):
  `yarn <script>` -> `npm run <script>`, `yarn --cwd X Y` ->
  `npm run Y --prefix X`, args passed via `--`
- tests.yml path filter: yarn.lock/.yarnrc -> package-lock.json/.npmrc
- type-check.yml: plugin cache key hashes package-lock.json; root .bin
  is still exposed on PATH (npm --prefix only surfaces the sub-dir bin)

Edit-and-review: validate on a test push (CI not runnable locally).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Dockerfile: COPY package-lock.json; root install `npm ci
  --ignore-scripts && npx patch-package` (skip vite optimize, keep
  patches); api installs via `npm ci [--omit=dev] --prefix`
- build.sh: `yarn` -> `npm ci`, build scripts -> `npm run`
- build_modules.sh: prod installs -> `npm ci --omit=dev`; the
  post-jq-surgery install uses `npm install --omit=dev` (lockfile
  intentionally out of sync then); restore package-lock.json
- build-statics.cmd: plugin installs/build -> npm (Windows twin of .sh)
- generate-api-client.js: spawn npm instead of yarn
- remove redisinsight/api/.yarnclean.prod

Dropped `yarn autoclean` (no npm equivalent) in Docker + build_modules;
release artifacts/image are slightly larger. Flagged in-file as a
node-prune follow-up. Edit-and-review: validate on a test build.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- api test.Dockerfile: COPY package-lock.json, `RUN npm ci`,
  CMD `npm run test:api:ci:cov` (postinstall still applies patches;
  client gen stays skipped via SKIP_API_CLIENT_GEN=1)
- tests/e2e static-server.Dockerfile: `yarn add express` ->
  `npm install express`
- postinstall-generate-client.js: retry message + comment reference npm

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- docs/npm-migration-plan.md: the full yarn->npm migration plan, rules,
  and findings (single self-contained reference)
- scripts/lock-diff.js: compares yarn.lock vs package-lock.json to detect
  removed/added packages; exits non-zero on genuine removals so it can
  gate a migration

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Convert all yarn command examples to npm (dev/test/lint/type-check/
tscheck), and rewrite the dependency/lockfile-management section: npm
install + package-lock.json, CI runs npm ci, npm install/uninstall for
dependency changes. Drops the obsolete yarn-deduplicate postinstall note.
CLAUDE.md is a symlink to AGENTS.md, so both are covered.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Convert yarn command references to npm across:
- CONTRIBUTING.md, .ai/skills/* (type-check-baselines, i18n, code-quality,
  tsconfigs, redis-insight-plugin + references/templates), .ai/commands
- plugin + api/i18n/e2e READMEs
- config-comment hints (i18next.config.mjs, .gitignore, .eslintignore,
  openapi-ts.config.ts)

`yarn --cwd X` -> `npm run ... --prefix X`, `yarn <script>` ->
`npm run <script>`, `yarn install --frozen-lockfile` -> `npm ci`,
`yarn add` -> `npm install`. Workspace-binary note reworded for npm
(root .bin still needed on PATH). Intentional ".npmrc mirrors yarn's
lenient resolution" rationale comments left as-is.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The re-st, re-clu, re-crdt, and oss-st-6-tls-auth-ssh integration-test
compose files ran the API test suite via a wait-for-it command array
ending in ['yarn', 'test:api:ci:cov']; switch to
['npm', 'run', 'test:api:ci:cov'].

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Leftovers found by a repo-wide audit (no lockfile hand-edits):
- install-all-build-libs action: remove the obsolete "verify yarn.lock
  unchanged after install" step (it guarded yarn-deduplicate's in-place
  rewrite; npm ci never mutates the lockfile and yarn.lock is gone)
- postinstall-generate-client.js: spawn `npm run generate:api-client`
  instead of `yarn generate:api-client`
- .vscode/launch.json: debug API config runs via npm
- .gitignore x3: drop yarn-debug/error.log patterns (npm-debug.log kept)
- decisions doc: document why lockfile `resolved` URLs keep
  registry.yarnpkg.com (faithful seed artifact; do not hand-edit)

Lockfiles are intentionally left as `npm install` produced them; the
registry.yarnpkg.com URLs are functional and must not be hand-swapped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This reverts commit 4ce478e.

The migration execution plan and the one-off lock-diff verification
script are process artifacts and don't belong in production code. Also
drop the now-dangling npm-migration-plan.md link from the decisions doc.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@pawelangelow pawelangelow self-assigned this Jul 16, 2026
pawelangelow and others added 2 commits July 16, 2026 18:10
The Docker builds copied package.json + package-lock.json but not
.npmrc, so `npm ci` ran without legacy-peer-deps=true and failed with
ERESOLVE on @elastic/eui@34.6.0's legacy react@^16 peer vs React 18
(main Dockerfile), and would hit the same on the api test image
(@mochajs/json-file-reporter). Copy .npmrc before the root/api npm ci.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Don't enumerate specific packages/peers in the Dockerfile comment; just
note that legacy peer resolution is required.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Code Coverage - Backend unit tests

St.
Category Percentage Covered / Total
🟢 Statements 93.14% 16504/17720
🟡 Branches 75.33% 5273/7000
🟢 Functions 87.47% 2533/2896
🟢 Lines 92.99% 15781/16970

Test suite run success

3686 tests passing in 321 suites.

Report generated by 🧪jest coverage report action from 6c033ce

@github-actions

github-actions Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Code Coverage - Integration Tests

Status Category Percentage Covered / Total
🟡 Statements 79.24% 18208/22976
🟡 Branches 61.87% 8484/13711
🟡 Functions 66.9% 2462/3680
🟡 Lines 78.85% 17142/21740

@github-actions

github-actions Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Code Coverage - Frontend unit tests

St.
Category Percentage Covered / Total
🟢 Statements 83.24% 27744/33332
🟡 Branches 69.25% 11904/17189
🟡 Functions 78.19% 7387/9448
🟢 Lines 83.69% 27015/32280

Test suite run success

7829 tests passing in 865 suites.

Report generated by 🧪jest coverage report action from 6c033ce

@pawelangelow
pawelangelow marked this pull request as ready for review July 16, 2026 17:51
@pawelangelow
pawelangelow requested a review from a team as a code owner July 16, 2026 17:52

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0600efe140

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread package.json
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant