Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CBG-4389: extract cv from known revs and store backup rev by cv #7237

Merged
merged 6 commits into from
Dec 11, 2024

Conversation

gregns1
Copy link
Contributor

@gregns1 gregns1 commented Dec 9, 2024

CBG-4389

  • Extract cv from known revs
  • Disable delta sync for revTree clients
  • Switch to backup rev by cv, using hash of cv for concerns about key length
  • Move backup for current rev to post update hlv function as due to macro expansion we don't have a version for cv at the backup rev function

Pre-review checklist

  • Removed debug logging (fmt.Print, log.Print, ...)
  • Logging sensitive data? Make sure it's tagged (e.g. base.UD(docID), base.MD(dbName))
  • Updated relevant information in the API specifications (such as endpoint descriptions, schemas, ...) in docs/api

Integration Tests

@gregns1 gregns1 changed the title CBG-4389: extract cv from knwon revs and store backup rev by revID CBG-4389: extract cv from knwon revs and store backup rev by cv Dec 9, 2024
@gregns1 gregns1 changed the title CBG-4389: extract cv from knwon revs and store backup rev by cv CBG-4389: extract cv from known revs and store backup rev by cv Dec 9, 2024
@gregns1 gregns1 requested a review from adamcfraser December 9, 2024 17:27
Copy link
Collaborator

@adamcfraser adamcfraser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few questions/comments.

@@ -1082,6 +1084,9 @@ func (bh *blipHandler) processRev(rq *blip.Message, stats *processRevStats) (err
if deltaSrcRevID, isDelta := revMessage.DeltaSrc(); isDelta && !revMessage.Deleted() {
if !bh.sgCanUseDeltas {
return base.HTTPErrorf(http.StatusBadRequest, "Deltas are disabled for this peer")
} else if !bh.useHLV() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For an incoming delta that has a revTreeID as deltaSrc, we should be able to accept if when deltaSrc is the current revTreeID. That case shouldn't require any temporary revision bodies.

Copy link
Contributor Author

@gregns1 gregns1 Dec 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point I think as long as the delta source rev is in the rev cache its fine, I think it makes sense for me to remove this upon reflection. If we receive a rev delta source, look in rev cache for it and fail to find it (then fail to load as no backup by rev id), the function will error anyway on line 1110.

db/blip_sync_context.go Outdated Show resolved Hide resolved
if bsc.useHLV() {
msgHLV, err := extractHLVFromBlipMessage(revID)
if err != nil {
base.ErrorfCtx(ctx, "Invalid known rev format for hlv on doc: %s", docID)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we can't parse this, should we fall back to sending full body instead of erroring? Or will it cause problems elsewhere?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On a related note, does the loop in line 374 also need to handle HLV format, or does that happen elsewhere?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good spot! Have updated to reflect this, let me know what you think!

db/crud.go Show resolved Hide resolved
Copy link
Collaborator

@adamcfraser adamcfraser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -2611,6 +2605,24 @@ func postWriteUpdateHLV(doc *Document, casOut uint64) *Document {
if doc.HLV.CurrentVersionCAS == expandMacroCASValueUint64 {
doc.HLV.CurrentVersionCAS = casOut
}
// backup new revision to the bucket now we have a doc assigned a CV (post macro expansion) for delta generation purposes
backupRev := db.deltaSyncEnabled() && db.deltaSyncRevMaxAgeSeconds() != 0
if db.UseXattrs() && backupRev {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the UseXattrs() check is unnecessary here (we don't support HLV with non-xattrs)

@adamcfraser adamcfraser merged commit 8668774 into release/anemone Dec 11, 2024
36 checks passed
@adamcfraser adamcfraser deleted the CBG-4389-new branch December 11, 2024 00:59
bbrks pushed a commit that referenced this pull request Dec 17, 2024
* CBG-4389: extract cv from knwon revs and store backup rev by revID

* update comments

* fix backup revs

* further tidy up

* udpated to address comments and fix flaking tests

* fix incorrect fetch format by cv in getCurrentVersion
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants