Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

consecutive negotiation #855

Merged
merged 12 commits into from
May 23, 2023
Merged

consecutive negotiation #855

merged 12 commits into from
May 23, 2023

Conversation

Byron
Copy link
Member

@Byron Byron commented May 18, 2023

Implement the consecutive negotiation algorithm to be more compatible to other kinds of servers which
cannot deal with the naive negotiation style we currently implement.

Tasks for git-negotiate

  • gix-negotiate crate with basic API
  • a test to see what happens if negotiation needs multiple rounds for baselines of HAVEs and COMMON and implement noop.
  • implementation consecutive
  • implementation for skipping
  • make consecutive ✅ and skipping ✅ actually work
  • add remaining test that also adds known refs for more coverage
  • make tests work with --dry-run - wasn't idempotent after all and it only works the first time it's run. There is some state apparently, despite --dry-run.

Tasks for gix

  • port multi-round test
  • support for adding tips of alternate dbs.
  • use negotiator and assure there is an object cache available for the graph traversal. Note that handle should not retry on misses (and do the same for describe). Assure we feed it dereferenced (i.e. peel to commit) commits only.

Research

  • Naive is really the first part of the 'common' algorithm which sends most of the refs as specified by the refspec. What's sent next is implemented by the algorithm, which we don't really implement at all.
  • with git fetch --negotiate-only it's possible to setup baseline expectations as we see the common commits produced by an algorithm, given a specific input of --negotiation-tips, which is equivalent to controlling the starting point of the negotiation
  • noop truly does nothing and is supposed to be used for refetching certain objects if a filter was used previously. It by itself wouldn't even trigger a full pack to be fetched as it also won't put any want line.
  • mark_complete_and_common_ref, mark_tips and for_each_cached_alternate (add tips of alternates) is the same sequence both in V1 and V2, then it sense HAVE lines as produced by the negotiator, feed ACK lines to it.
  • the first batch is definitely 16, but the window size adapts also depending on the protocol version (to be investigated)
  • --negotiate-only does its very own negotiation and thus isn't comparable - probably for V1 compatibility, we want to packet line tracing. However, negotiate_using_fetch provides a clean overview of what negotiation should be.
  • The window size for HAVE lines starts at 16, and is then increased with each round according to the rules in next_flush.
  • When traversing the commit-graph, which we probably have to do to some extend, the use of generation numbers is optional (and it defaults to INFINITY which effectively turns them off).

Byron added 2 commits May 19, 2023 11:16
…v0.1.0, safety bump 7 crates

SAFETY BUMP: gix-revision v0.14.0, gix-negotiate v0.1.0, gix v0.45.0, gix-refspec v0.11.0, gitoxide-core v0.28.0, cargo-smart-release v0.20.0, gitoxide v0.26.0
That way it's more similar to `gix_odb::at()` as there is effectively only one
important type.
@Byron Byron force-pushed the consecutive-negotiation branch 2 times, most recently from 7f90c8d to 009df44 Compare May 20, 2023 20:13
@Byron Byron force-pushed the consecutive-negotiation branch 2 times, most recently from 100beba to a219fa0 Compare May 22, 2023 19:20
Byron added 6 commits May 23, 2023 17:57
* make `CommitterTimestamp` available as type, making the code using it more descriptive.
* add `new()` to `PriorityQueue`
* add `Graph::try_lookup_and_insert_default()`
* add `Debug` impl for `Graph`
This is the default negotiation algorithm.
…istic fetch with --dry-run.

This means we will have to reproduce what git does naturally, to fill in common refs
and also provide tips.

Unfortunately this doesn't work as it's apparently not really dry-running, but modifying
the repository underneath. This means it's not idempotent when running it multiple times.
@Byron Byron merged commit 97b3f7e into main May 23, 2023
@Byron Byron deleted the consecutive-negotiation branch May 23, 2023 18:42
@Byron Byron mentioned this pull request May 23, 2023
13 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant