add a roadmap for the next 6 months #1784
Conversation
marten-seemann
requested review from
BigLep,
MarcoPolo,
p-shahi and
julian88110
marten-seemann
changed the title
ROADMAP.md
Outdated
|
|
||
| ## 🧠 Smart Dialing | ||
|
|
||
| Why: Having a large list of transports to pick from is great. Having an advanced stack that can dial all of them is even greater. But dialing all of them at the same time wastes our, the network’s and the peer’s resources. |
There was a problem hiding this comment.
| Why: Having a large list of transports to pick from is great. Having an advanced stack that can dial all of them is even greater. But dialing all of them at the same time wastes our, the network’s and the peer’s resources. | |
| Why: Having a large list of transports to pick from is great. Having an advanced stack that can dial all of them is even greater. But dialing all of them at the same time wastes the network’s and the peer’s resources. |
There was a problem hiding this comment.
It also wastes our CPU cycles to start that many handshakes.
|
This is awesome, my preliminary questions for now are:
|
We didn't, I did ;) I just wrote down what I think would be the most valuable to work on in the next half a year. So this is just my opinion. This list is intended as a starting point for the roadmap conversation, and shouldn't be viewed as a conclusion that was reached behind closed doors (it really isn't, I made up this entire list without asking for anybody's opinion on it). I'm happy to add and remove items based on feedback on this PR.
Everything below "Peer Routing Records" (including) from our general libp2p roadmap didn't make it. In addition to that, I've not included the part of Protocol Select that would replace multistream (libp2p/specs#349), since we're getting most of the benefit of the Protocol Select proposal by just doing the address change and the early muxer negotiation.
I was debating putting flaky tests on the list. We'll also engage with any bug reports and community PRs that come in, but I'm not sure if that needs a roadmap item or if it's just understood that we'll spend a certain amount of time on those.
We'd need to assign priorities to sub-items. For example, the early muxer negotiation from the Handshake section has higher priority than the item right on top (putting the security protocol in the multiaddr). How would we capture that?
That's a good question. We have to keep in mind that we're losing 1 - 1.5 months due to Lab Week, Thanksgiving and Xmas. My hope is that we can at least make significant progress on all of these points, but I might be too optimistic here. |
| @@ -0,0 +1,58 @@ | |||
| # go-libp2p roadmap Q4’22/Q1’23 | |||
There was a problem hiding this comment.
Great write up!
A general thought is that as libp2p is getting more widely deployed maybe we also need to think about beef up more security features. May or may not fit in to the next 6 months time frame.
For example behavior based security. The peer is usually not backed by a root of trust, so the traditional identity and chain of trust based approach does not work in this scenario. We may need to consider new approaches to enforce security.
There was a problem hiding this comment.
@julian88110 Do you have anything specific in mind?
Would that belong into the go-libp2p roadmap, or would that first and foremost be an item for the general libp2p roadmap (https://github.com/libp2p/specs/blob/master/ROADMAP.md)?
There was a problem hiding this comment.
Yes this should belong to the general roadmap for libp2p. I have some thoughts, may take some time to write it up after the muxer thing is done.
I'd like to see benchmarks as a section here. I think it's inline with future-proof testing as it's easy for things to get slow if you're not tracking them. I'll probably have some other comments, but I need to think about this a bit more. |
|
@MarcoPolo Good point! I added a benchmarking section in 60c4fce. How does that look to you? |
There was a problem hiding this comment.
@marten-seemann: thanks for getting the ball rolling here. This is good stuff!
@marten-seemann @p-shahi ;
I know this a dump of milestone definitions in no particular order and that there is more work happening in https://www.notion.so/pl-strflt/go-libp2p-22-Q4-23-Q1-Product-Roadmap-48bc31fd5b394a29812d6cda624fffa0 for faster iteration which will makes it way out hee.
I'll leave my current thoughts from what I'm seeing here so it's public and tied to the milestone definitions. This is a drive by because I'm not sure when I'll be able to circle back. I expect manh of these things are on your radar. Don't feel the need to respond to each comment. This is a dump while the thoughts were there for me.
- I like how "Comprehensive Metrics" has a tracking issue. I think that's useful for communicating if/where there are dependencies between milestone and it allows someone to follow along with progress and or ask questions. When we settle here, I think it would be good have these for the other items.
- It seems like some of these things are themes like "Universal Browser Connectivity" and that some items within them are milestones on their own ("WebTransport", "WebRTC"). I think having a way to index into the specific milestone within a theme is good/useful. That seems to be what Prithvi is doing here with "A1.1", "B1.2". etc. Great.
- I know this is coming, but we ultimately need to give expected dates (month targets ideally). A list broken out by month (sort of like what's in https://www.notion.so/pl-strflt/Milestone-Roadmap-a90d888d6bf14b999131ea7a802c9960 ) would be great. Not required, but showing which milestones are being developed in a given month is nice as well to set expectation on when progress will begin.
- On the surface "Judicious Address Advertisements" seems like a stretch for the next 6 months and the lowest priority. I'll be interested to see where that lands on the calendar estimate.
- Should we bring Futureproof testing to the forefront? I'm viewing some it as necessary for "Handshakes at the Speed of Light" to be completed and IPDX has laid a good foundation to build on from what they did in libp2p/test-plans. New/exciting work is still going to be happening with landing WebRTC (with LBL driving this). Part of me wonders if we'll be happy for "eating our veggies" so we have a solid foundation for 2023.
General roadmap hygiene/logistics:
- When this gets merged, lets make sure we have a link from "libp2p vision ROADMAP"
- Please also expose within go-libp2p README itself.
- Make sure we have a way for users to ask questions or comment. I worry that "open a PR to ask a question or make a suggestion" will prevent people from engaging.
- Set expectations on when this was last edited and when it will next be reviewed. Maybe there is a tracking issue for the next review we can link to give the community visibility. That would also be a place for people to leave comments.
ROADMAP.md
Outdated
|
|
||
| How: | ||
|
|
||
| 1. implement some kind of “Happy-Eyeballs” style prioritization among all supported transports |
There was a problem hiding this comment.
| 1. implement some kind of “Happy-Eyeballs” style prioritization among all supported transports | |
| 1. implement some kind of “[Happy-Eyeballs](https://en.wikipedia.org/wiki/Happy_Eyeballs)” style prioritization among all supported transports |
ROADMAP.md
Outdated
|
|
||
| Why: Having lots of transports is great, and shaving off RTTs is awesome. We need to stay backwards-compatible with legacy go-libp2p implementations, and less advanced libp2p stacks. | ||
|
|
||
| Goal: We have cross-version and cross-implementation Testground coverage that makes sure that we are able to establish a libp2p connection between two nodes, in the expected number of RTTs. This makes sure that the optimizations don’t break compatibility, actually have the performance impact we expect them to have, and serves as a regression test in the future. |
There was a problem hiding this comment.
We talk about ensuring compability between versions and implementations, but I assume there is an extra dimension we need to add around transports (and likely encryption and muxer).
Basically want to show
<go-libp2p, vX, QUIC> can talk with <go-libp2p, vY, QUIC>
<go-libp2p, vX, TCP> can talk with <go-libp2p, vY, TCP>
<go-libp2p, vX, WebTransport> can talk with <go-libp2p, vY, WebTranport>
etc.
Here is my (@mxinden) proposal for a roadmap for the next 6 months for the rust-libp2p maintainers (@elenaf9, @thomaseizinger and @mxinden) working on the rust-libp2p project. (rust-)libp2p is an open source and open community project. Thus: - I want to make sure we listen to the community and align our work accordingly. I invite **everyone** to leave feedback here. - The below is only planning work for the maintainers, not the community. Folks from the community may suggest and pick up work not prioritized (e.g. [Kademlia efficient querying](#kademlia-efficient-querying)) nor mentioned on this roadmap and we (maintainers) should be supporting them as best as we can. I consider this more of a short-lived document, enabling everyone working and using rust-libp2p to discuss and eventually align on a common mission, making prioritization explicit. Once done, it will likely be outdated quickly and thus I suggest never actualy merge it. In my eyes, continued project management is better done via GitHub issues and thus all the information here should eventually make it into the corresponding GitHub issues if not already the case. Also see: - [libp2p roadmap](https://github.com/libp2p/specs/blob/master/ROADMAP.md) - [go-libp2p roadmap](libp2p/go-libp2p#1784) The items in this roadmap are ordered by importance where I define importance as: - Enable others to build with rust-libp2p before building components ourself. - e.g.[cross behaviour communication](#cross-behaviour-communication) over[Kademlia client mode](#kademlia-client-mode) where the former enables the latter - Invest into tools that enable us to work more efficiently - e.g.[Testground](#testground-abstraction) and [release process](#release-process) - Better do one thing well than many things close to right. - e.g. deprioritize [BitSwap implementation](#bitswap-implementation) and continue on [QUIC](#quic-support) even once first ieration is merged. That does not imply that we won't support community members implementing it. - Improve existing components before introducing new ones. - e.g.[Kademlia client mode](#kademlia-client-mode) before [WebTransport](#webtransport) I am curious what folks think and again encourage everyone to share their thoughts.
Here is my (@mxinden) proposal for a roadmap for the next 6 months for the rust-libp2p maintainers (@elenaf9, @thomaseizinger, @jxs and @mxinden) working on the rust-libp2p project. (rust-)libp2p is an open source and open community project. Thus: - I want to make sure we listen to the community and align our work accordingly. I invite **everyone** to leave feedback here. - The below is only planning work for the maintainers, not the community. Folks from the community may suggest and pick up work not prioritized (e.g. [Kademlia efficient querying](#kademlia-efficient-querying)) nor mentioned on this roadmap and we (maintainers) should be supporting them as best as we can. I consider this more of a short-lived document, enabling everyone working and using rust-libp2p to discuss and eventually align on a common mission, making prioritization explicit. Once done, it will likely be outdated quickly and thus I suggest never actualy merge it. In my eyes, continued project management is better done via GitHub issues and thus all the information here should eventually make it into the corresponding GitHub issues if not already the case. Also see: - [libp2p roadmap](https://github.com/libp2p/specs/blob/master/ROADMAP.md) - [go-libp2p roadmap](libp2p/go-libp2p#1784) The items in this roadmap are ordered by importance where I define importance as: - Enable others to build with rust-libp2p before building components ourself. - e.g.[cross behaviour communication](#cross-behaviour-communication) over[Kademlia client mode](#kademlia-client-mode) where the former enables the latter - Invest into tools that enable us to work more efficiently - e.g.[Testground](#testground-abstraction) and [release process](#release-process) - Better do one thing well than many things close to right. - e.g. deprioritize [BitSwap implementation](#bitswap-implementation) and continue on [QUIC](#quic-support) even once first ieration is merged. That does not imply that we won't support community members implementing it. - Improve existing components before introducing new ones. - e.g.[Kademlia client mode](#kademlia-client-mode) before [WebTransport](#webtransport) I am curious what folks think and again encourage everyone to share their thoughts.
|
Followup comments are in the followup PR: #1804 (review) |
There was a problem hiding this comment.
@p-shahi and @marten-seemann : thanks for the continued work here. A couple of additional high level things:
-
This doesn't need to be solved now, but I would love to solve surfacing key tenets of libp2p.
I personally view these as security, stability (we don't regress with new versions), and performance (we don't regress between releases and there is insignificant performance difference with options like HTTP). I know those aren't tenets we've formalized in libp2p itself and more discussion needs to be had, but maybe it's worth us stating them for the go-libp2p project itself and we can discuss hoisting them up to libp2p as a whole. -
what are the thoughts on how we handle open comments that won't be easily resolvable? We'll want to get this PR merged, but presumably we'll do so without incorporating all the feedback. I want us to be able to make incremental progress without needing to keep PRs open indefinitely.
| # Roadmap Appendix | ||
| ## [A.⚡️Handshakes at the Speed of Light](https://github.com/libp2p/go-libp2p/issues/1807) | ||
|
|
||
| **Why**: Historically, libp2p has been very wasteful when it comes to round trips spent during connection establishment. This is slowing down our users, especially their TTFB (time to first byte) metrics. |
There was a problem hiding this comment.
I agree this is true. I would love to have more insight on the actual impact. For example, it would be great to have anecdotes from users on how much this is an adoption blocker.
This is more general comment. I don't think there's necessarily anything to do right now. It's more just to push to find ways to quantify problems in terms of impact as much as possible.
I do agree this issue rolls into one of our key pillars of "performance". TTFB performance isn't the existential threat I have heard complaints about though (vs. libp2p is slower than HTTP). I could be tuned in here though. Getting user input would be great.
There was a problem hiding this comment.
I believe TTFB is an important metric to both Saturn and ipfs cluster folks.
| #### End of Q4 (December) | ||
| - B.1 🧠 Smart Dialing - **Happy Eyeballs** | ||
| - C 📊 Comprehensive Metrics - **Swarm metrics** | ||
| - D.1 📺 Universal Browser Connectivity - **[WebRTC Browser -> Server](https://github.com/libp2p/go-libp2p/pull/1655)** |
There was a problem hiding this comment.
I think the ordering should be:
- D.1 📺 Universal Browser Connectivity - WebRTC Browser -> Server
- B.1 🧠 Smart Dialing - Happy Eyeballs
- C 📊 Comprehensive Metrics - Swarm metrics
Reason being:
- We're not going to do anything to slow down the browser connectivity story. This closes up the story as well on "all modern browsers can connect with a go-libp2p node".
- Smart dialing - assuming we actually hitting a breaking point for what transport to use
There was a problem hiding this comment.
Per comment other places I believe, I would bump smart dialing and comprehensive metrics until we're more solid on future proofing interoperability and performance.
|
|
||
| **How**: This will look similar to how we already expose resource manager metrics. Metrics can be added incrementally for libp2p’s components. First milestone is having metrics for the swarm. | ||
|
|
||
| ## [D. 📺 Universal Browser Connectivity](https://github.com/libp2p/go-libp2p/issues/1811) |
There was a problem hiding this comment.
I know the ordering of milestones is defined above. Maybe in a future PR we move the themes around to be in priority order. For example, as a theme, I think browser connectivity should be before all of these. I would likely put future proof testing at the top.
|
|
||
| **Goal**: We have cross-version and cross-implementation Testground coverage that makes sure that we are able to establish a libp2p connection between two nodes, in the expected number of RTTs. This makes sure that the optimizations don’t break compatibility, actually have the performance impact we expect them to have, and serves as a regression test in the future. | ||
|
|
||
| 1. RTT test for Early Muxer Negotiation (A.1) |
There was a problem hiding this comment.
I'm less convinced this warrants its own milestone. This is basically part of the done criteria for early muxer integration negotiation. All new functionality needs to be testing correctly as part of marking it complete. I think it should be moved up.
I think we need:
Comprehensive interoperability testing across:
libp2p implementations (already happening)
implementation versions (already happening)
transports (I don't think we're exhaustive)
security/encryption (I don't think we're exhaustive)
muxers (I don't think we're exhaustive)
That is the key thing we want I believe so we can move forward with confidence that we're not breaking anything.
| - A.1⚡Handshakes at the Speed of Light - **[Early Muxer Negotiation](https://github.com/libp2p/specs/issues/426)** | ||
| - E.1 🧪 Future-proof testing - **RTT test for Early Muxer Negotiation** |
There was a problem hiding this comment.
Per below, I think these should be combined.
| - E.1 🧪 Future-proof testing - **RTT test for Early Muxer Negotiation** | ||
|
|
||
| ### Mid Q4 (November) | ||
| - F.1 ⏱ Future-proof Benchmarking - **[Using nix-builders](https://github.com/testground/testground/pull/1425)** |
There was a problem hiding this comment.
I think before we spin up benchmarking, lets get interoperability nailed. The pieces are in place for that already with libp2p/test-plans. I discussed this more below.
Open comments will be captured in #1806 so that we can merge this PR and make incremental updates |
Co-authored-by: Steve Loeppky <stvn@loeppky.com>
I took a first stab at our roadmap for the next half a year. This document contains that are specific to the go-libp2p stack. General libp2p project items are listed in the roadmap in the specs repo. Sections are in no particular order.
This document is intended as a conversation starter.
I intend to keep this PR open for at least a week to give plenty of time for feedback.