Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change Bitbucket PR hook to point the source branch, commit & ref #3965

Merged
merged 2 commits into from
Jul 23, 2024

Conversation

j04n-f
Copy link
Contributor

@j04n-f j04n-f commented Jul 23, 2024

Description

This is the first fix for: #3932

Change the Pull Request hook parser to return the source commit, branch, and ref instead of the destination. Right now, the workflow pulls the destination configuration and code. It should pull the source configuration and code to verify that the configuration and code work as expected before merging the changes.

In case of the close event, the hook parser returns the destination branch, ref and merge commit. Usually, the contributor automatically deletes the source branch after merging the changes to the destination branch. Using the source values will cause the workflow to fail.

After the changes, Woodpecker will correctly download the workflow from the source branch (Pull Request commit), but it will fail to clone the repository. This issue is related to the commit format returned by the Bitbucket webhook. This inconsistency has already been reported: https://jira.atlassian.com/browse/BCLOUD-21201. The webhook returns a short SHA. The problem is that the git fetch command requires the full SHA.

A workaround for this issue is to use the ref to fetch the code:

clone:
  git:
    image: woodpeckerci/plugin-git
    settings:
      ref: ${CI_COMMIT_REF}

This is not ideal, because the Pull Request head won't always match the workflow commit, but it solves 80% of the event use cases (e.g. trigger a pull request workflow on change). This workaround won't work when re-running a previous workflow pointing to another commit, it will pull the last commit, not the previous one.

Solutions

The solution proposed by the community is to retrieve the full SHA from the Bitbucket API using the short one. This solution has drawbacks:

  • The Bitbucket API rate limit is 1000 req/h. This solution will reduce the maximum number of workflow runs per hour.
  • It requires a braking change in the forges interface because the ´Hook(...)´ method does not have an instance of the HTTP Client.

We propose to allow the git plugin to fetch the source code from a URL. The Bitbucket returns a link pointing to the commit.

This proposal only requires a small change to the git plugin:

if p.Pipeline.CommitLink != "" {...}

Git commands:

$ git fetch --no-tags --depth=1 --filter=tree:0 https://bitbucket.org/workspace/repo/commits/692972aabfec
$ git reset --hard -q 692972aabfec # It works with the short SHA

Woodpecker will set CommitLink to a blank string for the other forges, but Bitbuckket will use the one returned by the webhook.

@6543 6543 added bug Something isn't working forge/bitbucket bitbucket forge related labels Jul 23, 2024
@6543 6543 merged commit 047eb19 into woodpecker-ci:main Jul 23, 2024
6 of 7 checks passed
6543 pushed a commit that referenced this pull request Jul 23, 2024
)

## Description

This is the first fix for: #3932

Change the Pull Request hook parser to return the source commit, branch, and ref instead of the destination. Right now, the workflow pulls the destination configuration and code. It should pull the source configuration and code to verify that the configuration and code work as expected before merging the changes.

In case of the close event, the hook parser returns the destination branch, ref and merge commit. Usually, the contributor automatically deletes the source branch after merging the changes to the destination branch. Using the source values will cause the workflow to fail.

After the changes, Woodpecker will correctly download the workflow from the source branch (Pull Request commit), but it will fail to clone the repository. This issue is related to the commit format returned by the Bitbucket webhook. This inconsistency has already been reported: https://jira.atlassian.com/browse/BCLOUD-21201. The webhook returns a short SHA. The problem is that the `git fetch` command requires the full SHA. 

A workaround for this issue is to use the ref to fetch the code:

```yaml
clone:
  git:
    image: woodpeckerci/plugin-git
    settings:
      ref: ${CI_COMMIT_REF}
```

This is not ideal, because the Pull Request head won't always match the workflow commit, but it solves 80% of the event use cases (e.g. trigger a pull request workflow on change). This workaround won't work when re-running a previous workflow pointing to another commit, it will pull the last commit, not the previous one.

## Solutions

The solution proposed by the community is to retrieve the full SHA from the Bitbucket API using the short one. This solution has drawbacks:
- The Bitbucket API rate limit is 1000 req/h. This solution will reduce the maximum number of workflow runs per hour.
- It requires a braking change in the forges interface because the ´Hook(...)´ method does not have an instance of the HTTP Client. 

We propose to allow the git plugin to fetch the source code from a URL. The Bitbucket returns a link pointing to the commit. 

This proposal only requires a small change to the git plugin:
- Add a new optional parameter (e.g. CommitLink)
- Add a clause to the following conditional: https://github.com/woodpecker-ci/plugin-git/blob/7ac9615f409b539486b8841bd5ef01ae16bbc434/plugin.go#L79C1-L88C3
```go
if p.Pipeline.CommitLink != "" {...}
```
Git commands:
```shell
$ git fetch --no-tags --depth=1 --filter=tree:0 https://bitbucket.org/workspace/repo/commits/692972aabfec
$ git reset --hard -q 692972aabfec # It works with the short SHA
```
Woodpecker will set CommitLink to a blank string for the other forges, but Bitbuckket will use the one returned by the webhook.
@woodpecker-bot
Copy link
Collaborator

@woodpecker-bot woodpecker-bot mentioned this pull request Jul 23, 2024
1 task
6543 pushed a commit to woodpecker-ci/plugin-git that referenced this pull request Jul 25, 2024
## Description
This relates to woodpecker-ci/woodpecker#3965
and closes woodpecker-ci/woodpecker#3932.

The Bitbucket Pull Request Hook returns a short commit SHA. The git
fetch command requires a full-length SHA. Add a clause to detect when
the SHA1 has less than 40 characters. Fetch & switch the branch to reset
the expected commit.

Raise an error if the branch value is not defined.

## Test

Run script:

```shell
export GOOS=linux
export GOARCH=amd64
export CGO_ENABLED=0
export GO111MODULE=on
export CI_REPO_CLONE_URL=https://{User}:{Token}@bitbucket.org/{Workspace}/{Project}.git
export CI_WORKSPACE=./woodpecker
export CI_BUILD_EVENT=push
export CI_COMMIT_SHA=692972aabfec
export CI_COMMIT_BRANCH=test

./release/linux/amd64/plugin-git
```
Output:

```shell
+ git init -b test
Initialised empty Git repository in /home/pc415/bit2me/plugin-git/woodpecker/.git/
+ git config --global --replace-all safe.directory ./woodpecker
+ git remote add origin https://{User}:{Token}@bitbucket.org/{Workspace}/{Project}.git
+ git fetch --no-tags --depth=1 --filter=tree:0 origin +test:
warning: filtering not recognized by server, ignoring
From https://bitbucket.org/{Workspace}/{Project}.git
 * branch            test       -> FETCH_HEAD
 * [new branch]      test       -> origin/test
+ git switch -q test
+ git reset --hard -q 692972aabfec
+ git submodule update --init --recursive --depth=1 --recommend-shallow
+ git lfs fetch
```
6543 pushed a commit to 6543-forks/woodpecker that referenced this pull request Sep 5, 2024
…odpecker-ci#3965)

## Description

This is the first fix for: woodpecker-ci#3932

Change the Pull Request hook parser to return the source commit, branch, and ref instead of the destination. Right now, the workflow pulls the destination configuration and code. It should pull the source configuration and code to verify that the configuration and code work as expected before merging the changes.

In case of the close event, the hook parser returns the destination branch, ref and merge commit. Usually, the contributor automatically deletes the source branch after merging the changes to the destination branch. Using the source values will cause the workflow to fail.

After the changes, Woodpecker will correctly download the workflow from the source branch (Pull Request commit), but it will fail to clone the repository. This issue is related to the commit format returned by the Bitbucket webhook. This inconsistency has already been reported: https://jira.atlassian.com/browse/BCLOUD-21201. The webhook returns a short SHA. The problem is that the `git fetch` command requires the full SHA. 

A workaround for this issue is to use the ref to fetch the code:

```yaml
clone:
  git:
    image: woodpeckerci/plugin-git
    settings:
      ref: ${CI_COMMIT_REF}
```

This is not ideal, because the Pull Request head won't always match the workflow commit, but it solves 80% of the event use cases (e.g. trigger a pull request workflow on change). This workaround won't work when re-running a previous workflow pointing to another commit, it will pull the last commit, not the previous one.

## Solutions

The solution proposed by the community is to retrieve the full SHA from the Bitbucket API using the short one. This solution has drawbacks:
- The Bitbucket API rate limit is 1000 req/h. This solution will reduce the maximum number of workflow runs per hour.
- It requires a braking change in the forges interface because the ´Hook(...)´ method does not have an instance of the HTTP Client. 

We propose to allow the git plugin to fetch the source code from a URL. The Bitbucket returns a link pointing to the commit. 

This proposal only requires a small change to the git plugin:
- Add a new optional parameter (e.g. CommitLink)
- Add a clause to the following conditional: https://github.com/woodpecker-ci/plugin-git/blob/7ac9615f409b539486b8841bd5ef01ae16bbc434/plugin.go#L79C1-L88C3
```go
if p.Pipeline.CommitLink != "" {...}
```
Git commands:
```shell
$ git fetch --no-tags --depth=1 --filter=tree:0 https://bitbucket.org/workspace/repo/commits/692972aabfec
$ git reset --hard -q 692972aabfec # It works with the short SHA
```
Woodpecker will set CommitLink to a blank string for the other forges, but Bitbuckket will use the one returned by the webhook.
@woodpecker-bot woodpecker-bot mentioned this pull request Sep 8, 2024
1 task
@j04n-f j04n-f deleted the fix-bb-pr-hook-parse branch December 9, 2024 13:36
@woodpecker-bot woodpecker-bot mentioned this pull request Dec 14, 2024
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working forge/bitbucket bitbucket forge related
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants