CBG-3254: CBL pull replication for v4 protocol

[gregns1]

CBG-3254

Breakdown of changes:

• Changes to sendChanges: if in v4 protocol replication we populate the rev field on changes row with the current version off the change entry. For all other protocol versions we leave it as is.
• handleChangesResponse: added code to temporarily send full revision for v4 replication
  Inside sendRevision, I firstly changed revid -> rev given this can be a string of a revid or current version. Added code to fetch the doc by CV if running in v4 replication and above. Also added function toHistoryForHLV which will take a full HLV and convert it to the CBL string format needed for the replicator. The result of this is passed into blipRevMessageProperties as it was before.
• I needed to change the documentRevision struct returned by the rev cache. Before we just had CV on this struct but given we need the full HLV to send in a blip message inside sendRevision I have removed CV added a string representation of history to the stuct.
• I also enabled the versionVector sub test on the btcRunner. In doing this we needed a way to assert on the current version as well as the revid on the DocVersion struct created by Tor. So I have had to add temp functions to perform operations directly on the db (not through rest api) so it can be assigned to this struct. In doing this, inside GetVersion which is used to assert on in the blip tests I have added a lookup by CV
• I have added a test assertion function to assert on blip history entitled AssertOnBlipHistory which has to check what version the protocol is in as its very possible a lot of the time the history section of the blip message in our current test suite is empty given we are updating docs on the same source.
• Added a test entitled TestHLVMapToCBLString that test the format of the conversion of the HLV maps to CBL string format. For the test case that has both PV and MV defined I have had to split the resulting string and assert on substring as the resulting string can have Merge versions in a random order given iteration through a map is random.
• Added a test entitled TestPullReplicationUpdateOnOtherHLVAwarePeer which purpose is to see if blip history property is correctly populated in HLV aware replication. It makes use of Adams HLV agent to simulate a write from a different peer. In using this I had to move the HLV agent code into a different file as it was inaccessible in its old location
• I also refactored some of the btcRunner code from a conversion with Ben. We need the ability to skip sub tests in the btcRunner object. For example, the test I added TestPullReplicationUpdateOnOtherHLVAwarePeer needs to be HLV only replication. I have done this by passing in a map of the sub test you want to skip, this is getting used to skip attachment test and delta sync tests for CBL replication too (given we haven't yet implemented that). This allows us the freedom to add further sub tests to the runner in future (can imagine a world we may want a separate sub test for delta sync?)

Pre-review checklist

• Removed debug logging (fmt.Print, log.Print, ...)
• Logging sensitive data? Make sure it's tagged (e.g. base.UD(docID), base.MD(dbName))
• Updated relevant information in the API specifications (such as endpoint descriptions, schemas, ...) in docs/api

Dependencies (if applicable)

• Link upstream PRs
• Update Go module dependencies when merged

Integration Tests

• GSI=true,xattrs=true https://jenkins.sgwdev.com/job/SyncGateway-Integration/2269/

[adamcfraser]

CreateVersionFromString is a relatively expensive operation (base64 decoding and hex cas conversion). It's particularly true here, since these are coming from changeArray (which we just constructed when we sent a batch of changes).
More broadly, I feel like we should discuss whether we can have the majority of our code work with the formatted (base64 and string cas) in memory, to avoid constant conversion back and forth.

[gregns1]

I have made some changes (now to store on docRev the history in string format etc). I haven't yet removed this CreateVersionFromString() here. After the update to the storage format discussed on MB-60281 I started to change the way we represent the HLV in memory but stopped as it will grow this PR by a lot. I propose filing ticket and getting into next sprint to refactor to support the updated storage format and only convert to unit64 etc when doing conflict checks.

[adamcfraser]

Overall LGTM, two minor tweaks requested

[adamcfraser]

strings.Join is an efficient way to iterate over a set and do WriteString of separator and elements, but in this case we're already iterating over the set, and already have our own strings.Builder. Why not just add elements to s as we iterate over MergeVersions and PreviousVersions, instead of iterating twice?

[gregns1]

I agree in hindsight this makes more sense. I originally went for this approach to avoid adding a trailing ',' at the end of the string as this would break the design CBL expect. I have got round that now with a item counter so we can keep track of the elements in the map we are iterating over.

Transfered review ownership