Azure devops by souleb · Pull Request #216 · fluxcd/go-git-providers

souleb · 2023-04-28T15:11:52Z

Description

New azure devops implementation.

It can effectively reconcile azuredevops repositories using the azure zsk. It uses a PAT as ssh keys are not supported.

Commits and Pull requests are supported but teamspermissions are still to be implemented.

cc @nellyk

Test results

https://github.com/fluxcd/go-git-providers/actions/runs/4832004149/jobs/8610205652

nellyk · 2023-05-03T12:00:46Z

 	github.com/hashicorp/go-multierror v1.1.1
 	github.com/hashicorp/go-retryablehttp v0.7.2
 	github.com/ktrysmt/go-bitbucket v0.9.48
+	github.com/microsoft/azure-devops-go-api/azuredevops/v6 v6.0.1


latest release is v7 https://github.com/microsoft/azure-devops-go-api/releases/tag/azuredevops%2Fv7.1.0

we can't use V7 until they fix ContinuationToken field type. It is unconsistent depending on which struct you use.

@souleb is it related to this issue? microsoft/azure-devops-go-api#97 did you find it in a different struct?

i've mentioned this to the relevant team and will let you know when it gets resolved

Co-authored-by: souleb <bah.soule@gmail.com> Signed-off-by: Nelly Kiboi <nellykiboi@gmail.com>

Signed-off-by: Soule BA <soule@weave.works> Signed-off-by: nellyk <3062772+nellyk@users.noreply.github.com>

Signed-off-by: nellyk <3062772+nellyk@users.noreply.github.com>

Co-authored-by: souleb <bah.soule@gmail.com> Signed-off-by: Nelly Kiboi <3062772+nellyk@users.noreply.github.com> Signed-off-by: nellyk <3062772+nellyk@users.noreply.github.com>

Signed-off-by: nellyk <3062772+nellyk@users.noreply.github.com>

Co-authored-by: souleb <bah.soule@gmail.com> Signed-off-by: Nelly Kiboi <3062772+nellyk@users.noreply.github.com> Signed-off-by: nellyk <3062772+nellyk@users.noreply.github.com>

Signed-off-by: nellyk <3062772+nellyk@users.noreply.github.com>

Signed-off-by: Soule BA <soule@weave.works>

Also cleaning up and refactoring where needed. Signed-off-by: Soule BA <soule@weave.works>

codecov-commenter · 2023-05-11T10:43:00Z

Codecov Report

❌ Patch coverage is 60.93089% with 277 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.00%. Comparing base (f57ef5c) to head (f3bf893).
⚠️ Report is 147 commits behind head on main.

Files with missing lines	Patch %	Lines
azuredevops/client_repository_pullrequest.go	20.43%	73 Missing and 1 partial ⚠️
azuredevops/client_repository_tree.go	0.00%	58 Missing ⚠️
azuredevops/resource_repository.go	65.46%	42 Missing and 6 partials ⚠️
azuredevops/util.go	64.15%	15 Missing and 4 partials ⚠️
azuredevops/client_repositories.go	78.20%	13 Missing and 4 partials ⚠️
azuredevops/auth.go	51.61%	10 Missing and 5 partials ⚠️
azuredevops/azuredevopsclient.go	64.70%	11 Missing and 1 partial ⚠️
azuredevops/client_organizations.go	72.09%	9 Missing and 3 partials ⚠️
azuredevops/client_repository_commit.go	87.23%	7 Missing and 5 partials ⚠️
azuredevops/resource_pullrequest.go	82.14%	4 Missing and 1 partial ⚠️
... and 2 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #216      +/-   ##
==========================================
- Coverage   59.48%   59.00%   -0.49%     
==========================================
  Files          44       88      +44     
  Lines        3019     6891    +3872     
==========================================
+ Hits         1796     4066    +2270     
- Misses       1038     2251    +1213     
- Partials      185      574     +389

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

makkes · 2023-06-01T16:09:17Z

 | Test team         | fluxcd-test-team             | `STASH_TEST_TEAM_NAME`      |

+
+### Azure DevOps


Suggested change

### Azure DevOps

#### Azure DevOps

makkes · 2023-06-01T16:15:26Z

+	// No default value at POST-time.
+	// Added for Azure devops compatibility since it allows updating the name
+	// +optional
+	Name *string `json:"name"`


This type is also returned by e.g. UserRepository.Get. We should adapt the other implementations to properly fill this field. Otherwise it will be nil which feels strange, esp. since the information is available from the API types.

makkes · 2023-06-01T16:17:35Z

+
+const (
+	azureTokenFile     = "/tmp/azure.token"
+	defaultDescription = "Foo description"


This variable is only ever consumed once and it's not very important to declare it here (as opposed to the variable above). We should just inline it. Alternatively give it a name that make it easier to determine its intended use.

makkes · 2023-06-01T16:18:14Z

+)
+
+const (
+	azureTokenFile     = "/tmp/azure.token"


Optional: inline this value.

makkes · 2023-06-01T16:21:28Z

+	testing.Init()
+	rand.Seed(time.Now().UnixNano())


Is it really necessary to do this in an init func as opposed to TestProvider or in the azure Provider spec? I personally avoid init wherever possible as it's really hard to control or discover when navigating the code.

makkes · 2023-06-01T16:33:03Z

+		// Create if not found
+		if errors.Is(err, gitprovider.ErrNotFound) {
+			resp, err := c.Create(ctx, ref, req, toCreateOpts(opts...)...)
+			return resp, true, err


Suggested change

return resp, true, err

return resp, true, fmt.Errof("failed creating repository %q: %w", ref, err)

makkes · 2023-06-01T16:33:28Z

+)
+
+const (
+	emptyObjectPlaceholder = "0000000000000000000000000000000000000000"


This definitely needs an explanatory comment. What is it used for and why do we need it?

makkes · 2023-06-01T16:34:32Z

+	gitRefPrefix           = "refs/heads/"
+)
+
+// BranchClient implements the gitprovider.BranchClient interface.


Suggested change

// BranchClient implements the gitprovider.BranchClient interface.

I think this is self-explanatory to a Go programmer.

makkes · 2023-06-01T16:36:48Z

+	repositoryId := c.ref.GetRepository()
+	projectId := c.ref.GetIdentity()
+	ref := gitRefPrefix + branch
+	all := make([]any, 0, len(files))


Suggested change

all := make([]any, 0, len(files))

allFiles := make([]any, 0, len(files))

?

makkes · 2023-06-01T16:37:56Z

+	requests := make([]gitprovider.PullRequest, len(*prs))
+
+	for idx, pr := range *prs {
+		requests[idx] = newPullRequest(c.clientContext, &pr)


Suggested change

requests[idx] = newPullRequest(c.clientContext, &pr)

pr := pr

requests[idx] = newPullRequest(c.clientContext, &pr)

This is needed because Go reuses variables. You may end up with requests pointing to the same address for all items in the slice.

souleb requested review from makkes and yiannistri April 28, 2023 15:23

nellyk reviewed May 3, 2023

View reviewed changes

makkes reviewed May 5, 2023

View reviewed changes

Comment thread .github/workflows/e2e.yaml Outdated

nellyk and others added 26 commits May 10, 2023 18:28

add intial setup & get list of projects api

3a60af6

add intial setup & get list of projects api

7c0a256

adding a few tests and apis

08d5cc2

add get repository for project method for azure devops

dacd214

add get repository for project method for azure devops

8de69f7

address review comments

addba5d

remove space on auth file

a281f36

add support for Tree API

7ebb4c2

add create branch azuredevops

4e797ed

Update azuredevops/client_repository_tree.go

94bd68a

Co-authored-by: souleb <bah.soule@gmail.com> Signed-off-by: Nelly Kiboi <nellykiboi@gmail.com>

add intial setup & get list of projects api

b9a6ffd

add intial setup & get list of projects api

3d7c068

adding a few tests and apis

8f846ba

add get repository for project method for azure devops

25257c1

add get repository for project method for azure devops

a7ffc7c

address review comments

748da57

remove space on auth file

44c99a9

add support for Tree API

73d2cad

add create branch azuredevops

da06ba2

Add ability to create commits azure devops

218aa50

Add list commit azure devops

f1be3fe

Add ability to create pull request

65d258b

review comments

b09453d

Update azuredevops/client_repository_commit.go

aaad427

Co-authored-by: souleb <bah.soule@gmail.com> Signed-off-by: Nelly Kiboi <nellykiboi@gmail.com>

review comments

e80740b

review comments update commits function

f957e4a

nellyk and others added 19 commits May 10, 2023 18:31

commit review comments

577f59e

adding e2e tests for org repositories

97c8e28

Signed-off-by: Soule BA <soule@weave.works> Signed-off-by: nellyk <3062772+nellyk@users.noreply.github.com>

added repo list and fixing integration tests

2d93283

Signed-off-by: nellyk <3062772+nellyk@users.noreply.github.com>

Add ability to create repo

e3b554c

Signed-off-by: nellyk <3062772+nellyk@users.noreply.github.com>

intergration tests

8a3c83a

Signed-off-by: nellyk <3062772+nellyk@users.noreply.github.com>

fix integration tests

ab2b039

Signed-off-by: nellyk <3062772+nellyk@users.noreply.github.com>

remove space

8aad5eb

Co-authored-by: souleb <bah.soule@gmail.com> Signed-off-by: Nelly Kiboi <3062772+nellyk@users.noreply.github.com> Signed-off-by: nellyk <3062772+nellyk@users.noreply.github.com>

Update azuredevops/client_repositories.go

df7e404

Co-authored-by: souleb <bah.soule@gmail.com> Signed-off-by: Nelly Kiboi <3062772+nellyk@users.noreply.github.com> Signed-off-by: nellyk <3062772+nellyk@users.noreply.github.com>

update review comments

545900b

Signed-off-by: nellyk <3062772+nellyk@users.noreply.github.com>

update review comments

e2c87e1

Signed-off-by: nellyk <3062772+nellyk@users.noreply.github.com>

Update azuredevops/util.go

2b7d1b1

Co-authored-by: souleb <bah.soule@gmail.com> Signed-off-by: Nelly Kiboi <3062772+nellyk@users.noreply.github.com> Signed-off-by: nellyk <3062772+nellyk@users.noreply.github.com>

Update azuredevops/integration_suite_test.go

a9c2c92

Co-authored-by: souleb <bah.soule@gmail.com> Signed-off-by: Nelly Kiboi <3062772+nellyk@users.noreply.github.com> Signed-off-by: nellyk <3062772+nellyk@users.noreply.github.com>

update review comments

97994a4

Signed-off-by: nellyk <3062772+nellyk@users.noreply.github.com>

Add pull request methods

0ee0913

Signed-off-by: nellyk <3062772+nellyk@users.noreply.github.com>

review comments - add pr

c5b9980

Signed-off-by: nellyk <3062772+nellyk@users.noreply.github.com>

remove panic comments

975a68c

Signed-off-by: nellyk <3062772+nellyk@users.noreply.github.com>

adding missing DeployToken method

2e3b111

Signed-off-by: Soule BA <soule@weave.works>

adding azure env var to workflow

2256f08

Signed-off-by: Soule BA <soule@weave.works>

Adding azure devops projects tests

f3bf893

Also cleaning up and refactoring where needed. Signed-off-by: Soule BA <soule@weave.works>

souleb force-pushed the azure-devops branch from 8ba6009 to f3bf893 Compare May 11, 2023 10:39

makkes reviewed Jun 1, 2023

View reviewed changes

		\| Test team \| fluxcd-test-team \| `STASH_TEST_TEAM_NAME` \|


		### Azure DevOps

	return resp, true, err
	return resp, true, fmt.Errof("failed creating repository %q: %w", ref, err)

	all := make([]any, 0, len(files))
	allFiles := make([]any, 0, len(files))

	requests[idx] = newPullRequest(c.clientContext, &pr)
	pr := pr
	requests[idx] = newPullRequest(c.clientContext, &pr)

Conversation

souleb commented Apr 28, 2023

Description

Test results

Uh oh!

Choose a reason for hiding this comment

Uh oh!

souleb May 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecov-commenter commented May 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

souleb May 11, 2023 •

edited

Loading

codecov-commenter commented May 11, 2023 •

edited

Loading