Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[No QA] Fast fetch! #21651

Merged
merged 54 commits into from
Jul 10, 2023
Merged
Show file tree
Hide file tree
Changes from 50 commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
a1ffe07
Create utility functions to initialize and checkout repo w/ remote
roryabraham Jun 26, 2023
a0da5ba
Fix code that deletes all branches except main
roryabraham Jun 26, 2023
51ab8ce
Update utility functions to sync with the remote
roryabraham Jun 26, 2023
8aff00b
Create utility function for assertion
roryabraham Jun 26, 2023
a5e69b6
Set up github authentication
roryabraham Jun 26, 2023
bcab951
Remove gh auth token, use token instead
roryabraham Jun 26, 2023
a817c31
Setup authorization for remote
roryabraham Jun 27, 2023
b8dc0bb
Cleanup tags in addition to branches
roryabraham Jun 27, 2023
f10c02c
Make sure branches are created correctly
roryabraham Jun 27, 2023
ab52624
Silence red herring fatal error
roryabraham Jun 27, 2023
838fc31
Correctly fetch tags and create branches
roryabraham Jun 27, 2023
7bfc0c5
Use same semver schema as app
roryabraham Jun 27, 2023
aa59901
Use version updater from app
roryabraham Jun 27, 2023
a28fd12
Create getPreviousVersion utility
roryabraham Jun 28, 2023
2f8fae8
Update GitUtils to correctly fetch history
roryabraham Jun 28, 2023
8b565dc
Rebuild GH actions
roryabraham Jun 28, 2023
57b849e
Make constants readonly
roryabraham Jun 28, 2023
abdf8ee
Simplify git fetch calls
roryabraham Jun 28, 2023
437e678
Use --depth=1 in reassurePerformanceTests
roryabraham Jun 28, 2023
10acbf7
Add --force switch so that reassure doesn't fail if there are uncommi…
roryabraham Jun 28, 2023
50f5212
Fast fetch in e2ePerformanceTests
roryabraham Jun 28, 2023
e30c3e2
Fast fetch in cherryPick.yml
roryabraham Jun 28, 2023
6a8eb8e
Remove unnecessary fetch-depth: 0
roryabraham Jun 28, 2023
95c0c22
Remove unnecessary fetch-depth: 0 from platformDeploy
roryabraham Jun 28, 2023
bc73070
Remote fetch-depth from verifyPodfile
roryabraham Jun 28, 2023
3fd4701
Remove fetch-depth: 0 from deploy.yml
roryabraham Jun 28, 2023
431f30b
Remove unnecessary fetch-depth: 0 from deployBlocker
roryabraham Jun 28, 2023
bd1e9f8
Remove fetch-depth: 0 from validateDocsRoutes.yml
roryabraham Jun 28, 2023
94fe94d
Remove fetch-depth: 0 from preDeploy.yml
roryabraham Jun 28, 2023
74817ee
Remove fetch-depth: 0 from validateGitHubActions.yml
roryabraham Jun 28, 2023
8c1b389
Remove last fetch-depth: 0
roryabraham Jun 28, 2023
62a649c
Merge branch 'main' into Rory-FetchOnlyOneTag
roryabraham Jun 28, 2023
14c0f63
Update README with guidelines for fast fetch
roryabraham Jun 28, 2023
21e9982
Add warning if GITHUB_TOKEN not set
roryabraham Jun 28, 2023
c4b9368
Add concurrency key for shellTests to sync to the remote
roryabraham Jun 28, 2023
460b5b6
Create remote server locally
roryabraham Jun 28, 2023
61e2662
Remove check for GITHUB_TOKEN
roryabraham Jun 28, 2023
e501733
Add git config for initial commit
roryabraham Jun 28, 2023
3185e54
Fix JSDoc formatting
roryabraham Jun 28, 2023
9e549af
Add missing JSDoc
roryabraham Jun 28, 2023
e24f947
Fix variable name
roryabraham Jun 28, 2023
7a74e41
Reduce diff
roryabraham Jun 28, 2023
d396111
Inline remove_local_repo function
roryabraham Jun 28, 2023
389412b
Simplify update functions to match workflows
roryabraham Jun 28, 2023
b01f7c0
Merge branch 'main' into Rory-FetchOnlyOneTag
roryabraham Jul 7, 2023
ac9756f
Fetch merge_commit_sha in cherryPick.yml
roryabraham Jul 7, 2023
b87ca24
Conditional fetch of merge commit
roryabraham Jul 7, 2023
6db69d2
Remove unnecessary steps from cherryPick.yml
roryabraham Jul 7, 2023
825aceb
RIP triggerWorkflowAndWait
roryabraham Jul 7, 2023
3eb0eab
Merge branch main into Rory-FetchOnlyOneTag
roryabraham Jul 7, 2023
1966337
Fetch HEAD_COMMIT_SHA
roryabraham Jul 9, 2023
50825f7
Make fetch work in origin or fork
roryabraham Jul 9, 2023
e0e34df
Remove unnecessary conditional
roryabraham Jul 9, 2023
3bb11f8
Add explicit production ref checkout
roryabraham Jul 10, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions .github/actions/javascript/bumpVersion/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -272,6 +272,41 @@ const incrementVersion = (version, level) => {
return incrementPatch(major, minor, patch);
};

/**
* @param {String} currentVersion
* @param {String} level
* @returns {String}
*/
function getPreviousVersion(currentVersion, level) {
const [major, minor, patch, build] = getVersionNumberFromString(currentVersion);

if (level === SEMANTIC_VERSION_LEVELS.MAJOR) {
if (major === 1) {
return getVersionStringFromNumber(1, 0, 0, 0);
}
return getVersionStringFromNumber(major - 1, 0, 0, 0);
}

if (level === SEMANTIC_VERSION_LEVELS.MINOR) {
if (minor === 0) {
return getPreviousVersion(currentVersion, SEMANTIC_VERSION_LEVELS.MAJOR);
}
return getVersionStringFromNumber(major, minor - 1, 0, 0);
}

if (level === SEMANTIC_VERSION_LEVELS.PATCH) {
if (patch === 0) {
return getPreviousVersion(currentVersion, SEMANTIC_VERSION_LEVELS.MINOR);
}
return getVersionStringFromNumber(major, minor, patch - 1, 0);
}

if (build === 0) {
return getPreviousVersion(currentVersion, SEMANTIC_VERSION_LEVELS.PATCH);
}
return getVersionStringFromNumber(major, minor, patch, build - 1);
}

module.exports = {
getVersionNumberFromString,
getVersionStringFromNumber,
Expand All @@ -282,6 +317,7 @@ module.exports = {
SEMANTIC_VERSION_LEVELS,
incrementMinor,
incrementPatch,
getPreviousVersion,
};


Expand Down
185 changes: 166 additions & 19 deletions .github/actions/javascript/createOrUpdateStagingDeploy/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -216,29 +216,30 @@ const _ = __nccwpck_require__(3571);
const {spawn, execSync} = __nccwpck_require__(3129);
const CONST = __nccwpck_require__(4097);
const sanitizeStringForJSONParse = __nccwpck_require__(9338);
const {getPreviousVersion, SEMANTIC_VERSION_LEVELS} = __nccwpck_require__(8007);

/**
* @param {String} tag
*/
// eslint-disable-next-line no-unused-vars
function fetchTagIfNeeded(tag) {
function fetchTag(tag) {
const previousPatchVersion = getPreviousVersion(tag, SEMANTIC_VERSION_LEVELS.PATCH);
try {
console.log(`Checking if tag ${tag} exists locally`);
const command = `git rev-parse --verify ${tag}`;
console.log(`Running command: ${command}`);
const result = execSync(command).toString();
console.log(result);
} catch (e) {
console.log(`Tag ${tag} not found locally, attempting to fetch it.`);
let command = `git fetch origin tag ${tag} --no-tags`;

// Exclude commits reachable from the previous patch version (i.e: previous checklist),
// so that we don't have to fetch the full history
// Note that this condition would only ever _not_ be true in the 1.0.0-0 edge case
if (previousPatchVersion !== tag) {
command += ` --shallow-exclude=${previousPatchVersion}`;
}

console.log(`Running command: ${command}`);
let result = execSync(command).toString();
console.log(result);
console.log('Verifying that the tag is now available...');
command = `git rev-parse --verify ${tag}`;
execSync(command);
} catch (e) {
// This can happen if the tag was only created locally but does not exist in the remote. In this case, we'll fetch history of the staging branch instead
const command = `git fetch origin staging --no-tags --shallow-exclude=${previousPatchVersion}`;
console.log(`Running command: ${command}`);
result = execSync(command).toString();
console.log(result);
execSync(command);
}
}

Expand All @@ -250,10 +251,8 @@ function fetchTagIfNeeded(tag) {
* @returns {Promise<Array<Object<{commit: String, subject: String, authorName: String}>>>}
*/
function getCommitHistoryAsJSON(fromTag, toTag) {
// fetchTagIfNeeded(fromTag);
// fetchTagIfNeeded(toTag);
// Note: this is a temporary measure until we can figure out a faster way to fetch only what's needed
execSync('git fetch --all --tags');
fetchTag(fromTag);
fetchTag(toTag);

console.log('Getting pull requests merged between the following tags:', fromTag, toTag);
return new Promise((resolve, reject) => {
Expand Down Expand Up @@ -905,6 +904,154 @@ module.exports = function (inputString) {
};


/***/ }),

/***/ 8007:
/***/ ((module, __unused_webpack_exports, __nccwpck_require__) => {

const _ = __nccwpck_require__(3571);

const SEMANTIC_VERSION_LEVELS = {
MAJOR: 'MAJOR',
MINOR: 'MINOR',
PATCH: 'PATCH',
BUILD: 'BUILD',
};
const MAX_INCREMENTS = 99;

/**
* Transforms a versions string into a number
*
* @param {String} versionString
* @returns {Array}
*/
const getVersionNumberFromString = (versionString) => {
const [version, build] = versionString.split('-');
const [major, minor, patch] = _.map(version.split('.'), (n) => Number(n));

return [major, minor, patch, Number.isInteger(Number(build)) ? Number(build) : 0];
};

/**
* Transforms version numbers components into a version string
*
* @param {Number} major
* @param {Number} minor
* @param {Number} patch
* @param {Number} [build]
* @returns {String}
*/
const getVersionStringFromNumber = (major, minor, patch, build = 0) => `${major}.${minor}.${patch}-${build}`;

/**
* Increments a minor version
*
* @param {Number} major
* @param {Number} minor
* @returns {String}
*/
const incrementMinor = (major, minor) => {
if (minor < MAX_INCREMENTS) {
return getVersionStringFromNumber(major, minor + 1, 0, 0);
}

return getVersionStringFromNumber(major + 1, 0, 0, 0);
};

/**
* Increments a Patch version
*
* @param {Number} major
* @param {Number} minor
* @param {Number} patch
* @returns {String}
*/
const incrementPatch = (major, minor, patch) => {
if (patch < MAX_INCREMENTS) {
return getVersionStringFromNumber(major, minor, patch + 1, 0);
}
return incrementMinor(major, minor);
};

/**
* Increments a build version
*
* @param {Number} version
* @param {Number} level
* @returns {String}
*/
const incrementVersion = (version, level) => {
const [major, minor, patch, build] = getVersionNumberFromString(version);

// Majors will always be incremented
if (level === SEMANTIC_VERSION_LEVELS.MAJOR) {
return getVersionStringFromNumber(major + 1, 0, 0, 0);
}

if (level === SEMANTIC_VERSION_LEVELS.MINOR) {
return incrementMinor(major, minor);
}

if (level === SEMANTIC_VERSION_LEVELS.PATCH) {
return incrementPatch(major, minor, patch);
}

if (build < MAX_INCREMENTS) {
return getVersionStringFromNumber(major, minor, patch, build + 1);
}

return incrementPatch(major, minor, patch);
};

/**
* @param {String} currentVersion
* @param {String} level
* @returns {String}
*/
function getPreviousVersion(currentVersion, level) {
const [major, minor, patch, build] = getVersionNumberFromString(currentVersion);

if (level === SEMANTIC_VERSION_LEVELS.MAJOR) {
if (major === 1) {
return getVersionStringFromNumber(1, 0, 0, 0);
}
return getVersionStringFromNumber(major - 1, 0, 0, 0);
}

if (level === SEMANTIC_VERSION_LEVELS.MINOR) {
if (minor === 0) {
return getPreviousVersion(currentVersion, SEMANTIC_VERSION_LEVELS.MAJOR);
}
return getVersionStringFromNumber(major, minor - 1, 0, 0);
}

if (level === SEMANTIC_VERSION_LEVELS.PATCH) {
if (patch === 0) {
return getPreviousVersion(currentVersion, SEMANTIC_VERSION_LEVELS.MINOR);
}
return getVersionStringFromNumber(major, minor, patch - 1, 0);
}

if (build === 0) {
return getPreviousVersion(currentVersion, SEMANTIC_VERSION_LEVELS.PATCH);
}
return getVersionStringFromNumber(major, minor, patch, build - 1);
}

module.exports = {
getVersionNumberFromString,
getVersionStringFromNumber,
incrementVersion,

// For tests
MAX_INCREMENTS,
SEMANTIC_VERSION_LEVELS,
incrementMinor,
incrementPatch,
getPreviousVersion,
};


/***/ }),

/***/ 7351:
Expand Down
Loading