Feature: GitHub autofork url by balazsbajorics · Pull Request #4704 · concrete-utopia/utopia

balazsbajorics · 2024-01-08T16:32:16Z

Verbose URL https://utopia.pizza/p/?github_owner=balazsbajorics&github_repo=hydrogen-november&github_branch=main

Shorter URL https://utopia.pizza/p/?clone=balazsbajorics/hydrogen-november

Shorter URL with branch name https://utopia.pizza/p/?clone=balazsbajorics/hydrogen-november&github_branch=main

Try Me link:
Verbose URL link

Problem
The only (maintained, aside from the old zip-based repo checkout POC) working method for cloning a github repository was to create a new project, navigate to the github tab, then insert the owner / repo name, select a branch, press clone, overwrite contents of the new project.

We already had a URL-based entry point for the git checkout proof of concept which was cloning a git repo using a zip download. To my surprise it still works today, but it doesn't do the asset upload step, and does not connect the editor's github tab to the cloned repository. Plus I couldn't remember what other limitations it might have, so Balazs 2023 December decided to not spend time trying to understand if we should fix that instead of using the main github cloning feature.

What did I do
So I set out to replace the existing git auto-clone functionality with our "modern" and maintained github integration. I went down a bit of a dead end in #4660 and yesterday I decided to scrap most of my changes and come up with a vastly simplified solution.
Basically what this PR ended up doing is that it still does load (an empty) project, and then calls the same code path that runs when a user selects a repository in the github tab.

Flow of cloning a repo:
The flow is orchestrated from GithubRepositoryCloneFlow which is a react component in editor-component. This is currently our only loading flow which is not directly orchestrated from editor.tsx.

Editor.tsx detects that we want to load from a github repo, and sets the EditorStore accordingly
GithubRepositoryCloneFlow gets enabled, and makes sure the user is logged in and authenticated with github.
Once the user is logged in and authenticated, we fire off cloneGithubRepo
3.cloneGithubRepo creates a new totally empty project that only has a package.json. It fires PersistenceMachine.createNew so we get a proper Project ID, then saves this empty project to the server
the code awaits the LOAD action being dispatched to avoid race conditions
once the LOAD has been dispatched, we initiate the github cloning by calling GithubOperations.updateProjectWithBranchContent (that's the exact same code path that is called if the user clicks "Load from Branch" in the github pane)
Profit

Changes to the Server
Parallel to my editor changes, I also added a new server endpoint: the ability to clone a repo without specifying a branch. Since the default branch of git repos can be arbitrarily named, the server code first figures out the default repo branch name, and then calls the existing load-a-branch code path.

Not logged in states

If the user is not logged in, or not authenticated with github, we stop the flow and wait for their action.

github-actions · 2024-01-08T16:34:39Z

Try me

relativeci · 2024-01-08T16:40:34Z

Job #9890: Bundle Size — 62.25MiB (~+0.01%).

a5b7b24(current) vs df0030c master#9881(baseline)

Warning

Bundle contains 66 duplicate packages – View duplicate packages

Bundle metrics

4 changes

1 regression

1 improvement

	Current Job #9890	Baseline Job #9881
Initial JS	`45.44MiB`(`~+0.01%`)	`45.44MiB`
Initial CSS	`0B`	`0B`
Cache Invalidation	`25.65%`	`19.89%`
Chunks	`26`	`26`
Assets	`30`	`30`
Modules	`4365`(`+0.02%`)	`4364`
Duplicate Modules	`471`(`-0.21%`)	`472`
Duplicate Code	`30.76%`	`30.76%`
Packages	`462`	`462`
Duplicate Packages	`65`	`65`

Bundle size by type

1 change

1 regression

	Current Job #9890	Baseline Job #9881
JS	`62.24MiB` (`~+0.01%`)	`62.23MiB`
HTML	`11.54KiB`	`11.54KiB`

View job #9890 report View feature/github-fork-url branch activity

github-actions · 2024-01-08T16:52:53Z

Performance test results:

seanparsons · 2024-01-09T10:48:04Z

+          const githubBranch = urlParams.get('github_branch')
          const importURL = urlParams.get('import_url')

+          const githubRepoObj: GithubRepo | null = (() => {


Could this just be turned into a regular function and have it be invoked here? Feels like a lot of URL handling logic all embedded within the Editor component.

yeah very good point, it's became too massive

…flow

ruggi · 2024-01-11T16:56:57Z

  startGithubAuthentication: startGithubAuthentication,
 } as const
+
+export function useOnClickAuthenticateWithGithub() {


should this live in a separate .tsx file?

do you have a problem with the current location other than it's a ts file?

ruggi · 2024-01-12T10:02:29Z

+      'branches',
+      githubRepo.owner,
+      githubRepo.repository,
+      'default-branch',


shall we stick with GH's (unchanged) default here? (main)

I'm not sure what you mean. if you say we can assume the default branch is main, that's wrong.

seanparsons · 2024-01-12T10:38:22Z


 function createStoryboardFileWithPlaceholderContents(
  projectContents: ProjectContentTreeRoot,
+  isItNecessary: 'create-placeholder' | 'skip-creating-placeholder',


Be nice to extract this into a type definition as it appears to be in at least 2 places.

Also isItNecessary is a bad name. shouldCreatePlaceholder or just createPlaceholder would be better

I did the renaming, I did not create a type definition though because I think this is more readable. we should talk about it in the chat if you disagree :)

seanparsons · 2024-01-12T10:39:58Z

+    }),
+  ])
+
+  // TODO make sure the EditorState knows we have a github repo connected!!!


Future work?

yep, for the test I will write

seanparsons · 2024-01-12T10:41:01Z

+  return null
+})
+
+const awaitLoadActionDispatchedByPersistenceMachine = (): Promise<{ projectId: string }> => {


Can this be a regular function?

Rheeseyb · 2024-01-12T10:57:50Z

+
 export const DefaultPackageJson = {
-  name: 'Utopia Project',
+  name: 'utopia-project',


somewhat unrelated mini fix

Rheeseyb · 2024-01-12T11:00:42Z

+  githubRepo: GithubRepoWithBranch,
+) {
+  if (didWeInitiateGitRepoDownloadSinceTheEditorLoaded) {
+    return


Should this throw an error instead? This feels like something that shouldn't fail silently

this function gets called a couple of times because of ancestor remounts, maybe I should put a comment here

a much much simpler github autoforking

51109ef

Server: endpoint to default branch of a repo

05aef3d

balazsbajorics force-pushed the feature/github-fork-url branch from 1ee8773 to 05aef3d Compare January 8, 2024 16:36

balazsbajorics added 3 commits January 8, 2024 21:48

moving PubSub out of dispatch

2c4a5c7

Update editor-update.spec.tsx

d79d8d3

add option for shorter clone URL

257544c

balazsbajorics marked this pull request as ready for review January 9, 2024 10:38

seanparsons reviewed Jan 9, 2024

View reviewed changes

seanparsons approved these changes Jan 9, 2024

View reviewed changes

balazsbajorics added 16 commits January 9, 2024 18:13

a GithubRepositoryCloneFlow component to orchestrate the github load …

60b5953

…flow

moving code to file

6675a11

Update editor.tsx

6b39827

Merge branch 'master' into feature/github-fork-url

8330cad

new updatePubSubAtomFromOutsideReact

5e7c334

extracted SignInButton

8143e5a

extracted AuthenticateWithGithubButton

ddbdf8f

progress on the clone overlay

8cb04c4

v1 of the actual working clone flow

ceb4186

copy fixes and not showing double overlay during github load

9da7479

dispatching setGithubState in editor.tsx

4ec3d8b

making secondaryButton optional

8a70073

using Dialog for the github clone flow

9350b1e

Merge branch 'master' into feature/github-fork-url

0a54f9b

Do not create placeholder Storyboard file while loading from Github

dae4979

moving getGithubRepoToLoad to file

5fecf66

balazsbajorics requested a review from seanparsons January 11, 2024 15:54

undoing unrelated pubsub atom changes

6bc4a8d

balazsbajorics added 6 commits January 11, 2024 16:57

undoing unrelated editor.tsx changes

59a7f67

undoing more changes that ended up being irrelevant

a91429d

moving FullScreenOverlay to dedicated file

2d58ef2

actually undoing all these FullScreenOverlay changes

fae290f

Update github-clone-overlay.tsx

209bf8f

renaming GithubRepositoryCloneFlow

5bcaa3c

ruggi reviewed Jan 11, 2024

View reviewed changes

balazsbajorics added 2 commits January 11, 2024 23:20

using isLoggedIn

71704e5

Update github-repository-clone-flow.tsx

6638975

ruggi reviewed Jan 12, 2024

View reviewed changes

seanparsons reviewed Jan 12, 2024

View reviewed changes

seanparsons approved these changes Jan 12, 2024

View reviewed changes

Rheeseyb reviewed Jan 12, 2024

View reviewed changes

moving useOnClickAuthenticateWithGithub to its own file

cc2f706

Rheeseyb approved these changes Jan 12, 2024

View reviewed changes

ruggi approved these changes Jan 12, 2024

View reviewed changes

balazsbajorics added 3 commits January 14, 2024 14:46

Merge branch 'master' into feature/github-fork-url

20182a4

Update actions.tsx

e8e4470

Update github-repository-clone-flow.tsx

a5b7b24

balazsbajorics merged commit 15cd213 into master Jan 15, 2024

balazsbajorics deleted the feature/github-fork-url branch January 15, 2024 08:53

ruggi mentioned this pull request Feb 15, 2024

Remix: fork projects #4910

Merged

Uh oh!

Conversation

balazsbajorics commented Jan 8, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jan 8, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

relativeci Bot commented Jan 8, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Job #9890: Bundle Size — 62.25MiB (~+0.01%).

Uh oh!

github-actions Bot commented Jan 8, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ruggi Jan 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

balazsbajorics Jan 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

balazsbajorics commented Jan 8, 2024 •

edited

Loading

github-actions Bot commented Jan 8, 2024 •

edited

Loading

relativeci Bot commented Jan 8, 2024 •

edited

Loading

github-actions Bot commented Jan 8, 2024 •

edited

Loading

ruggi Jan 12, 2024 •

edited

Loading

balazsbajorics Jan 12, 2024 •

edited

Loading