Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: SITES-25634 [Importer] Expose hasCustomImportJs and hasCustomHeaders as two new fields on a import job #509

Merged
merged 3 commits into from
Sep 24, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@
"@adobe/helix-shared-wrap": "2.0.2",
"@adobe/helix-status": "10.1.3",
"@adobe/helix-universal-logger": "3.0.20",
"@adobe/spacecat-shared-data-access": "1.44.5",
"@adobe/spacecat-shared-data-access": "1.45.1",
"@adobe/spacecat-shared-http-utils": "1.6.10",
"@adobe/spacecat-shared-ims-client": "1.3.16",
"@adobe/spacecat-shared-rum-api-client": "2.9.4",
Expand Down
2 changes: 2 additions & 0 deletions src/dto/import-job.js
Original file line number Diff line number Diff line change
Expand Up @@ -30,5 +30,7 @@ export const ImportJobDto = {
successCount: importJob.getSuccessCount(),
failedCount: importJob.getFailedCount(),
redirectCount: importJob.getFailedCount(),
hasCustomHeaders: importJob.hasCustomHeaders(),
hasCustomImportJs: importJob.hasCustomImportJs(),
}),
};
21 changes: 18 additions & 3 deletions src/support/import-supervisor.js
Original file line number Diff line number Diff line change
Expand Up @@ -94,9 +94,19 @@ function ImportSupervisor(services, config) {
* @param {string} hashedApiKey - API key used to authenticate the import job request.
* @param {object} options - Client provided options for the import job.
* @param initiatedBy - Details about the initiator of the import job.
* @param {boolean} hasCustomHeaders - Whether custom headers are provided. Defaults to false.
* @param {boolean} hasCustomImportJs - Whether custom import JS is provided. Defaults to false.
* @returns {Promise<ImportJob>}
*/
async function createNewImportJob(urls, importQueueId, hashedApiKey, options, initiatedBy) {
async function createNewImportJob(
urls,
importQueueId,
hashedApiKey,
options,
initiatedBy,
hasCustomHeaders = false,
hasCustomImportJs = false,
) {
const newJob = {
id: crypto.randomUUID(),
baseURL: determineBaseURL(urls),
Expand All @@ -106,6 +116,8 @@ function ImportSupervisor(services, config) {
urlCount: urls.length,
status: ImportJobStatus.RUNNING,
initiatedBy,
hasCustomHeaders,
hasCustomImportJs,
};
return dataAccess.createNewImportJob(newJob);
}
Expand All @@ -126,6 +138,7 @@ function ImportSupervisor(services, config) {
* asynchronously.
* @param {Array<string>} urls - Array of URL records to queue.
* @param {object} importJob - The import job record.
* @param {object} customHeaders - Optional custom headers to be sent with each request.
*/
async function queueUrlsForImportWorker(urls, importJob, customHeaders) {
log.info(`Starting a new import job of baseUrl: ${importJob.getBaseURL()} with ${urls.length}`
Expand Down Expand Up @@ -159,8 +172,8 @@ function ImportSupervisor(services, config) {
* @param {string} importApiKey - The API key to use for the import job.
* @param {object} options - Optional configuration params for the import job.
* @param {string} importScript - Optional custom Base64 encoded import script.
* @param initiatedBy - Details about the initiator of the import job.
* @param customHeaders - Optional custom headers to be sent with each request.
* @param {object} initiatedBy - Details about the initiator of the import job.
* @param {object} customHeaders - Optional custom headers to be sent with each request.
* @returns {Promise<ImportJob>}
*/
async function startNewJob(
Expand All @@ -184,6 +197,8 @@ function ImportSupervisor(services, config) {
hashedApiKey,
options,
initiatedBy,
!!customHeaders,
!!importScript,
);

log.info('New import job created:\n'
Expand Down
Loading