feat: introduce audit for broken backlinks SITES-18417

[dzehnder]

PR Status

This PR implements the audit worker for SITES-18417.
Relates to adobe/spacecat-audit-post-processor#37

[iuliag]

Some points to be addressed below:

[solaris007]

Audit results need to be stored using dataAccess.addAudit. See https://github.com/adobe/spacecat-audit-worker/blob/main/src/lhs/handler.js#L265 for reference.

[solaris007]

We should audit on a site-level... audits need to be stored and referenced to a site. see https://github.com/adobe/spacecat-audit-worker/blob/main/src/lhs/handler.js#L353

[iuliag]

Audit results need to be stored using dataAccess.addAudit. See https://github.com/adobe/spacecat-audit-worker/blob/main/src/lhs/handler.js#L265 for reference.

@solaris007 , this is storing the audit result in DynamoDB, no? Is it mandatory or just best practice?

We should audit on a site-level... audits need to be stored and referenced to a site. see https://github.com/adobe/spacecat-audit-worker/blob/main/src/lhs/handler.js#L353

The url passed through message represents a site, no? Or here, do you actually want to point out that all those site and audit config validations in the given link need to be done as well?

[github-actions]

This PR will trigger a minor release when merged.

[solaris007]

@iuliag

this is storing the audit result in DynamoDB, no? Is it mandatory or just best practice?

I would consider it both mandatory and best-practice :) audits should be standardized (exceptions apply), in that we store them into our data layer for exposure to our various UIs (e.g. slack bot) and via the HTTP API, serving for analysis and visualization.

The url passed through message represents a site, no? Or here, do you actually want to point out that all those site and audit config validations in the given link need to be done as well?

the URL represents a site, if the url corresponds to a baseURL (unique for sites) for a known site. all audits (exceptions apply) should check whether the audit target (the site) exists and furthermore if the audit config of the site disables audits in general or specifically for that audit type.

Where can I check what they actually represent, e.g. what is siteId, fullAuditRef?

Please see the "data model" tab here: https://lucid.app/lucidchart/f5480d07-626d-42e8-8eb5-39b0e9a1c75f/edit?viewport_loc=-128%2C323%2C2452%2C1717%2C0_0&invitationId=inv_c77bd463-5df8-4973-b70b-a8028a6ee21a

Some details also in the readme (outdated): https://github.com/adobe/spacecat-shared/tree/main/packages/spacecat-shared-data-access

[basecode]

@solaris007 what of your review is blocking vs what can we do later? Before we aim for engineering excellence, I need to get the use case out in the wild to discuss how/if it makes sense. Thanks!

[solaris007]

@basecode the execution of the audit in the context of a site obtained from the data layer and storage of the audit result is not engineering excellence, but essential for downstream automation and consumption

[solaris007]

Create a scheduler that triggers once a week for specified sites

Weekly schedule (dev + prod) have been created with event payload { "type": "broken-backlinks" } for evaluation by the audit-scheduler lambda

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (c6db132) 100.00% compared to head (a517fd1) 100.00%.
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##              main       #77    +/-   ##
==========================================
  Coverage   100.00%   100.00%            
==========================================
  Files           10        12     +2     
  Lines         1111      1306   +195     
==========================================
+ Hits          1111      1306   +195     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Please re-review

[solaris007]

🎉 This PR is included in version 1.7.0 🎉

The release is available on:

• v1.7.0
• GitHub release

Your semantic-release bot 📦🚀