Add Draw mode for freehand screen annotation by aidenybai · Pull Request #473 · aidenybai/react-grab

aidenybai · 2026-06-16T06:07:46Z

Summary

Adds a Draw tool to the toolbar that lets you sketch freehand strokes and type text notes anywhere on the page, then copies a WYSIWYG screenshot (drawings included) to the clipboard.

Drawing: freehand strokes via vendored perfect-freehand (consolidated into a single file), pressure-aware, rendered on a fixed overlay canvas.
Text notes: type to drop a pink note at the cursor; click a committed note to re-edit it.
Shortcuts: Enter copies the screenshot, Escape discards in-progress text then exits, Cmd/Ctrl+Z undoes the last action.
Anchoring: strokes/text are stored in page coordinates and re-pinned on scroll/resize.
UI reuse: the Copy/Cancel menu and the "Copied" toast share a single AnchoredDropdownPanel (also adopted by the existing toolbar menu to kill duplication). Other toolbar actions are locked while Draw owns the screen.
Capture: only react-grab chrome (toolbar + menu) is hidden during getDisplayMedia; the drawing canvas stays visible so the shot is WYSIWYG. User-cancel of the share prompt is treated as a normal action; clipboard failures are logged.

Test plan

pnpm typecheck / pnpm lint / pnpm format / pnpm build clean
draw.spec.ts + toolbar-actions.spec.ts e2e pass (27/27)
Manual: draw + text + undo, Enter copies, Escape cancels, scroll keeps drawings pinned

Note

Medium Risk
Draw relies on browser screen-share and clipboard APIs with a large new interaction surface in core keyboard/routing; capture/session handling is hardened but permission UX and cross-browser support remain sensitive.

Overview
Adds a Draw tool so users can sketch freehand strokes and type text notes on the page, then copy a WYSIWYG viewport screenshot (annotations included) to the clipboard via getDisplayMedia and ClipboardItem.

Toolbar & API: New Draw button (hidden when screen capture or clipboard image write is unsupported), DRAW_ACTION_ID, drawPlugin with D shortcut, and api.draw() toggle. While drawing, copy/comment/style toolbar actions are disabled; Copy/Cancel menu and a tracked Copied toast use a new shared AnchoredDropdownPanel (toolbar menu refactored onto it). Default toolbar action falls back if Draw was saved as default on an unsupported browser.

Draw engine: createDrawModeController mounts a full-screen overlay/canvas with vendored perfect-freehand strokes, page-anchored geometry on scroll, inline text via a native <input>, undo, and session-safe capture that hides only react-grab chrome for the frame grab.

Tests: New draw.spec.ts e2e coverage for toolbar, API, shortcuts, menu, strokes, cancel/escape, undo, and scroll anchoring.

^{Reviewed by Cursor Bugbot for commit 59bc6cb. Bugbot is set up for automated code reviews on this repo. Configure here.}

Summary by cubic

Adds a Draw mode to react-grab so you can sketch freehand and add text notes on the page, then copy a WYSIWYG screenshot with annotations to the clipboard. Draw only appears (and api.draw() only runs) when screen capture and clipboard‑image write are supported; other tools and selection APIs are blocked while drawing.

New Features
- Freehand strokes and inline text on a fixed overlay canvas (pressure‑aware via vendored perfect-freehand), anchored to page coordinates on scroll/resize; text is edited in a native <input> and flattened on commit; click to re‑edit.
- Shortcuts: D to toggle, Enter to copy (exits if nothing drawn), Esc to cancel, Cmd/Ctrl+Z to undo; E2E covers the D shortcut.
- WYSIWYG capture: hide react‑grab chrome only for the frame; a “Copied” toast tracks the toolbar and is announced by screen readers.
- Toolbar Draw button and api.draw() toggle drawing; exported drawPlugin. The default‑action picker filters disabled actions and falls back if Draw was saved as default on an unsupported browser; other toolbar actions are disabled while drawing.
Bug Fixes
- Hardened capture: make onBeforeGrab synchronous, bound the post‑hide delay, paint the canvas synchronously before the frame, stop the capture stream on any failure, scope capture to a per‑session flag, restore hidden chrome on any exit, and skip the share prompt if the session ended.
- Input/keyboard: ignore IME composition; ignore printable keys mid‑stroke; while editing a note, only Enter (copy) and Esc (discard) are intercepted; ignore Cmd/Ctrl+Shift+Z (redo); gate the bare “D” shortcut where unsupported or in prompt mode; suppress the native context menu; disabled toolbar buttons are unfocusable.
- Drawing model: maintain one ordered list across strokes and text for correct undo and z‑order; ignore overlapping pointers; release pointer capture when clearing/undoing a live stroke; use real pen pressure for stroke width and simulate pressure only for mouse/touch; preserve true zero pen pressure.
- Text notes: re‑edit keeps the original on Esc and updates in place on Enter; reposition the open input on resize; invalidate cached width when re‑editing so the click‑to‑edit hit area stays accurate; placing a note after scrolling now drops it under the cursor even if the mouse didn’t move.
- Integration/UI: hide Draw where unsupported; filter context‑free‑disabled actions in the toolbar menu while keeping context‑dependent ones; clear the “Copied” toast when Draw reopens.

^{Written for commit 59bc6cb. Summary will update on new commits.}

Adds a toolbar Draw tool that lets you sketch freehand strokes and type text notes over the page, then copies a WYSIWYG screenshot to the clipboard (Enter to copy, Escape to cancel, Cmd/Ctrl+Z to undo). - Vendors perfect-freehand (consolidated, single file) for stroke output - Page-coordinate anchoring so drawings ride along with scroll/resize - Copy/Cancel menu + "Copied" toast share one AnchoredDropdownPanel - Other toolbar actions are locked while Draw owns the screen - e2e coverage in draw.spec.ts

vercel · 2026-06-16T06:07:52Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
react-grab-storybook	Ready	Preview, Comment	Jun 16, 2026 11:04am
react-grab-website	Ready	Preview, Comment	Jun 16, 2026 11:04am

pkg-pr-new · 2026-06-16T06:08:38Z

Open in StackBlitz

npm i https://pkg.pr.new/@react-grab/cli@473

npm i https://pkg.pr.new/grab@473

npm i https://pkg.pr.new/react-grab@473

commit: 59bc6cb

aidenybai · 2026-06-16T06:10:48Z

bugbot run

- Restore hidden toolbar/menu if the user exits draw mode mid-capture, and skip the screen-share prompt when no longer active after the capture delay - Ignore IME composition keydowns so confirming a composition doesn't trigger capture or corrupt an in-progress text note - Stop the getDisplayMedia stream on any capture failure (metadata/ready rejections previously leaked the stream)

aidenybai · 2026-06-16T06:16:32Z

bugbot run

cubic-dev-ai

3 issues found and verified against the latest diff

_{Reply with feedback, questions, or to request a fix.

Re-trigger cubic}

The toast computed its anchor once, so a post-capture reflow (e.g. the screen-share indicator bar collapsing) shifted the toolbar and stranded the toast at a stale position. Track the toolbar continuously for the toast's lifetime, mirroring the Copy/Cancel menu, and stop the tracker on fade-out and dispose.

- Unify committed strokes/texts into one ordered list so Cmd/Ctrl+Z undoes the genuinely-last action (and later items paint on top) regardless of type - Move all stream-touching setup inside the capture try/finally so a metadata/ready-state rejection can't leak the screen-capture stream - Add role="status" / aria-live="polite" to the "Copied" toast so screen readers announce it, matching CompletionView

aidenybai · 2026-06-16T06:24:09Z

bugbot run

cubic-dev-ai

1 issue found across 5 files (changes from recent commits).

_{Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic}

- Tag each draw session with an id and bail from an in-flight capture after any await if the session ended or was restarted, so a stale capture can no longer copy an abandoned shot or tear down a freshly-started session - Hide the Draw toolbar button and block the activation path (shortcut / api.annotate) when screen capture or clipboard-image write is unsupported, honoring the annotate plugin's existing enabled gate

aidenybai · 2026-06-16T07:28:41Z

bugbot run

Reopening Draw before the toast's fade timer fired left it tracking and visible during the new session. Centralize toast teardown in one helper and call it from onOpen (and dispose), so a new session always starts clean.

- Use event.pressure directly instead of `|| fallback`, which treated a valid pressure of 0 (lightest pen touch) as missing; PointerEvent.pressure is always a number, so the fallback (and its constant) were unnecessary - Rename the draw e2e describe block to "Draw mode" (was "Draw (draw) mode" after the annotate->draw rename)

aidenybai · 2026-06-16T10:06:26Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

1 issue from previous review remains unresolved.

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit bef1964. Configure here.}

Replace the canvas-drawn note + hand-rolled blinking caret with a focused <input> (native caret, selection, IME, blur-to-commit) positioned over the canvas; flatten it onto the canvas on commit so the screenshot stays WYSIWYG, and re-open an input when a committed note is clicked. The input is frameless (plain accent text + native caret), dropping the canvas caret/box drawing, the blink interval, the manual IME guard while editing, and their constants.

aidenybai · 2026-06-16T10:22:07Z

bugbot run

lastPointer held page coordinates captured at the last pointer event, so scrolling without moving the mouse and then typing dropped the note at the stale page position. Store the pointer in client (viewport) coords and add the current scroll only when placing the note.

aidenybai · 2026-06-16T10:28:36Z

bugbot run

Strokes recorded PointerEvent.pressure but getStroke ran with the default simulatePressure: true, ignoring it. Tag each stroke by input type so pen strokes use their recorded pressure while mouse/touch (constant pressure) keep the velocity-based simulation.

aidenybai · 2026-06-16T10:34:59Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit f2146da. Configure here.}

- Re-editing a committed note no longer deletes it on Escape: the note is tracked by index, hidden behind its input, and updated in place on commit or left untouched on discard (was spliced out and lost) - Paint the canvas synchronously before the screen grab so the capture can't race the scheduled redraw - Ignore Cmd/Ctrl+Shift+Z so redo doesn't trigger another undo - Reposition an open note input on resize, not just scroll - Hoist the shared pencil path to DRAW_PENCIL_PATH_D (used by the icon + cursor)

edit/comment/copy plugins inline their action id literal; only draw used the DRAW_ACTION_ID constant. Inline "draw" for consistency (the constant stays for the toolbar/core call sites, same as the others).

aidenybai · 2026-06-16T11:04:18Z

bugbot run

cursor

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 59bc6cb. Configure here.}

vercel Bot deployed to Preview – react-grab-storybook June 16, 2026 06:08 View deployment

vercel Bot deployed to Preview – react-grab-website June 16, 2026 06:08 View deployment

cursor Bot reviewed Jun 16, 2026

View reviewed changes

Comment thread packages/react-grab/src/core/draw-mode.ts

Comment thread packages/react-grab/src/core/draw-mode.ts

cursor Bot reviewed Jun 16, 2026

View reviewed changes

Comment thread packages/react-grab/src/utils/capture-screenshot.ts

vercel Bot deployed to Preview – react-grab-storybook June 16, 2026 06:16 View deployment

vercel Bot deployed to Preview – react-grab-website June 16, 2026 06:17 View deployment

cubic-dev-ai Bot reviewed Jun 16, 2026

View reviewed changes

Comment thread packages/react-grab/src/utils/capture-screenshot.ts

Comment thread packages/react-grab/src/core/annotation-mode.ts Outdated

Comment thread packages/react-grab/src/components/toolbar/annotation-copied-toast.tsx Outdated

cursor Bot reviewed Jun 16, 2026

View reviewed changes

Comment thread packages/react-grab/src/core/index.tsx Outdated

vercel Bot deployed to Preview – react-grab-storybook June 16, 2026 06:21 View deployment

vercel Bot deployed to Preview – react-grab-website June 16, 2026 06:21 View deployment

vercel Bot deployed to Preview – react-grab-storybook June 16, 2026 06:24 View deployment

vercel Bot deployed to Preview – react-grab-website June 16, 2026 06:24 View deployment

cursor Bot reviewed Jun 16, 2026

View reviewed changes

Comment thread packages/react-grab/src/core/draw-mode.ts

vercel Bot reviewed Jun 16, 2026

View reviewed changes

Comment thread packages/react-grab/src/components/toolbar/index.tsx Outdated

cubic-dev-ai Bot reviewed Jun 16, 2026

View reviewed changes

Comment thread packages/react-grab/src/core/annotation-mode.ts Outdated

vercel Bot deployed to Preview – react-grab-storybook June 16, 2026 07:28 View deployment

vercel Bot deployed to Preview – react-grab-website June 16, 2026 07:28 View deployment

cursor Bot reviewed Jun 16, 2026

View reviewed changes

Comment thread packages/react-grab/src/core/index.tsx

Clear the "Copied" toast when draw mode reopens

3b974d0

Reopening Draw before the toast's fade timer fired left it tracking and visible during the new session. Centralize toast teardown in one helper and call it from onOpen (and dispose), so a new session always starts clean.

cursor Bot reviewed Jun 16, 2026

View reviewed changes

Comment thread packages/react-grab/src/core/draw-mode.ts

vercel Bot deployed to Preview – react-grab-storybook June 16, 2026 10:06 View deployment

vercel Bot deployed to Preview – react-grab-website June 16, 2026 10:06 View deployment

cursor Bot reviewed Jun 16, 2026

View reviewed changes

vercel Bot reviewed Jun 16, 2026

View reviewed changes

Comment thread packages/react-grab/src/core/draw-mode.ts Outdated

vercel Bot deployed to Preview – react-grab-storybook June 16, 2026 10:22 View deployment

vercel Bot deployed to Preview – react-grab-website June 16, 2026 10:22 View deployment

cursor Bot reviewed Jun 16, 2026

View reviewed changes

Comment thread packages/react-grab/src/utils/get-draw-stroke-path.ts

vercel Bot deployed to Preview – react-grab-storybook June 16, 2026 10:30 View deployment

vercel Bot deployed to Preview – react-grab-website June 16, 2026 10:30 View deployment

cursor Bot reviewed Jun 16, 2026

View reviewed changes

Comment thread packages/react-grab/src/core/draw-mode.ts

vercel Bot deployed to Preview – react-grab-storybook June 16, 2026 10:35 View deployment

vercel Bot deployed to Preview – react-grab-website June 16, 2026 10:35 View deployment

cursor Bot reviewed Jun 16, 2026

View reviewed changes

vercel Bot deployed to Preview – react-grab-storybook June 16, 2026 11:03 View deployment

vercel Bot deployed to Preview – react-grab-website June 16, 2026 11:03 View deployment

Inline the draw action id to match the other plugins

59bc6cb

edit/comment/copy plugins inline their action id literal; only draw used the DRAW_ACTION_ID constant. Inline "draw" for consistency (the constant stays for the toolbar/core call sites, same as the others).

vercel Bot deployed to Preview – react-grab-storybook June 16, 2026 11:04 View deployment

vercel Bot deployed to Preview – react-grab-website June 16, 2026 11:04 View deployment

cursor Bot reviewed Jun 16, 2026

View reviewed changes

Comment thread packages/react-grab/src/components/renderer.tsx

Conversation

aidenybai commented Jun 16, 2026 • edited by cubic-dev-ai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Summary by cubic

Uh oh!

vercel Bot commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pkg-pr-new Bot commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aidenybai commented Jun 16, 2026

Uh oh!

Uh oh!

aidenybai commented Jun 16, 2026

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aidenybai commented Jun 16, 2026

Uh oh!

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

aidenybai commented Jun 16, 2026

Uh oh!

Uh oh!

Uh oh!

aidenybai commented Jun 16, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

aidenybai commented Jun 16, 2026

Uh oh!

aidenybai commented Jun 16, 2026

Uh oh!

Uh oh!

Uh oh!

aidenybai commented Jun 16, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

aidenybai commented Jun 16, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

aidenybai commented Jun 16, 2026 •

edited by cubic-dev-ai Bot

Loading

vercel Bot commented Jun 16, 2026 •

edited

Loading

pkg-pr-new Bot commented Jun 16, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading