Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(kad): add refresh_interval config used to poll bootstrap #4838

Merged
merged 45 commits into from
Mar 8, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
dc770aa
add poll function in kademlia behaviour that polls bootstrap function
PanGan21 Nov 12, 2023
30559da
bumb kad version and add changelog
PanGan21 Nov 12, 2023
a7fcd26
bumb server version and add server changelog
PanGan21 Nov 12, 2023
695b94a
Fix doc comments
PanGan21 Nov 12, 2023
0b54f9a
Rename to bootstrap_interval
PanGan21 Nov 12, 2023
d749cbd
Update protocols/kad/src/behaviour.rs
PanGan21 Nov 12, 2023
7f720ce
fix terminology across behaviour
PanGan21 Nov 12, 2023
3a7737c
move polling into the internal poll function
PanGan21 Nov 14, 2023
ac0fdd6
fix conflicts
PanGan21 Nov 14, 2023
7343bb2
do not reference private fields in doc
PanGan21 Nov 14, 2023
271ed03
Merge branch 'master' into kademlia-poll-bootstrap
PanGan21 Nov 23, 2023
17e941c
Merge branch 'master' into kademlia-poll-bootstrap
PanGan21 Nov 25, 2023
ea694e5
Merge branch 'master' into kademlia-poll-bootstrap
PanGan21 Nov 27, 2023
f492105
call bootstrap on new kademlia node connected
PanGan21 Nov 27, 2023
dcf8495
add enum to track the state of the initial bootstrap
PanGan21 Nov 28, 2023
e7595a0
Merge branch 'master' into kademlia-poll-bootstrap
PanGan21 Nov 28, 2023
72bc4be
Merge branch 'master' into kademlia-poll-bootstrap
PanGan21 Nov 29, 2023
f72a8a1
fix conflicts
PanGan21 Nov 30, 2023
f2aaad2
track if there was a successfull bootstrap and the query id of the au…
PanGan21 Nov 30, 2023
4d43c33
Merge branch 'master' into kademlia-poll-bootstrap
PanGan21 Dec 1, 2023
8604f72
fix conflicts
PanGan21 Dec 2, 2023
179db10
add link in doc, handle successful bootstrap, reanable polling
PanGan21 Dec 2, 2023
9c7b8e8
fix doc and remove test change
PanGan21 Dec 2, 2023
75889e1
Use defensive programming
thomaseizinger Dec 3, 2023
091cb10
Merge branch 'master' into kademlia-poll-bootstrap
thomaseizinger Dec 3, 2023
f14b988
Refactor using enum that represends the bootstrap status
PanGan21 Dec 5, 2023
a7d2bc0
fix existing unit tests
PanGan21 Dec 5, 2023
987b89a
Merge branch 'master' into kademlia-poll-bootstrap
PanGan21 Dec 5, 2023
f0e563e
satisfy clippy
PanGan21 Dec 5, 2023
82332a8
Merge branch 'master' into kademlia-poll-bootstrap
PanGan21 Dec 5, 2023
3aa9101
Slightly re-design `bootstrap::Status`
thomaseizinger Dec 6, 2023
c796150
fix conflicts
PanGan21 Jan 17, 2024
850bc22
feat(kad): Add periodic and automatic bootstrap
PanGan21 Nov 12, 2023
fe3e558
Merge pull request #1 from stormshield-frb/PanGan21-automatic-bootstrap
PanGan21 Jan 24, 2024
bb8ca9f
fix conflicts
PanGan21 Jan 25, 2024
99458a7
Preventing automatic_throttle configuration
stormshield-frb Feb 15, 2024
394d86b
Merge pull request #2 from stormshield-frb/PanGan21-automatic-bootstrap
PanGan21 Feb 20, 2024
b18c8d1
fix conflicts
PanGan21 Feb 20, 2024
fcafa60
satisfy clippy
PanGan21 Feb 20, 2024
cc2f533
Merge branch 'master' into kademlia-poll-bootstrap
PanGan21 Feb 25, 2024
89b011a
fix conflicts
PanGan21 Mar 3, 2024
245c2c8
fix: use current kad version
PanGan21 Mar 6, 2024
3000dd1
Merge branch 'master' into kademlia-poll-bootstrap
PanGan21 Mar 6, 2024
5f8ef81
reference automatic_bootstrap_throttle in the docs
PanGan21 Mar 7, 2024
a58ed37
fix bootstrap doc to omit private config
PanGan21 Mar 7, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ libp2p-floodsub = { version = "0.44.0", path = "protocols/floodsub" }
libp2p-gossipsub = { version = "0.46.0", path = "protocols/gossipsub" }
libp2p-identify = { version = "0.44.0", path = "protocols/identify" }
libp2p-identity = { version = "0.2.7" }
libp2p-kad = { version = "0.45.1", path = "protocols/kad" }
libp2p-kad = { version = "0.45.2", path = "protocols/kad" }
libp2p-mdns = { version = "0.45.0", path = "protocols/mdns" }
libp2p-memory-connection-limits = { version = "0.2.0", path = "misc/memory-connection-limits" }
libp2p-metrics = { version = "0.14.1", path = "misc/metrics" }
Expand All @@ -97,7 +97,7 @@ libp2p-quic = { version = "0.10.1", path = "transports/quic" }
libp2p-relay = { version = "0.17.0", path = "protocols/relay" }
libp2p-rendezvous = { version = "0.14.0", path = "protocols/rendezvous" }
libp2p-request-response = { version = "0.26.0", path = "protocols/request-response" }
libp2p-server = { version = "0.12.4", path = "misc/server" }
libp2p-server = { version = "0.12.5", path = "misc/server" }
libp2p-swarm = { version = "0.44.0", path = "swarm" }
libp2p-swarm-derive = { version = "=0.34.0", path = "swarm-derive" } # `libp2p-swarm-derive` may not be compatible with different `libp2p-swarm` non-breaking releases. E.g. `libp2p-swarm` might introduce a new enum variant `FromSwarm` (which is `#[non-exhaustive]`) in a non-breaking release. Older versions of `libp2p-swarm-derive` would not forward this enum variant within the `NetworkBehaviour` hierarchy. Thus the version pinning is required.
libp2p-swarm-test = { version = "0.3.0", path = "swarm-test" }
Expand Down
6 changes: 6 additions & 0 deletions misc/server/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
## 0.12.5
### Changed

- Use `Behaviour::poll` to call `Behaviour::bootstrap` for Kademlia protocol.
See [PR 4838](https://github.com/libp2p/rust-libp2p/pull/4838).

## 0.12.4

### Added
Expand Down
2 changes: 1 addition & 1 deletion misc/server/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "libp2p-server"
version = "0.12.4"
version = "0.12.5"
authors = ["Max Inden <mail@max-inden.de>"]
edition = "2021"
repository = "https://github.com/libp2p/rust-libp2p"
Expand Down
16 changes: 1 addition & 15 deletions misc/server/src/main.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
use base64::Engine;
use clap::Parser;
use futures::stream::StreamExt;
use futures_timer::Delay;
use libp2p::identity;
use libp2p::identity::PeerId;
use libp2p::kad;
Expand All @@ -14,17 +13,13 @@ use prometheus_client::registry::Registry;
use std::error::Error;
use std::path::PathBuf;
use std::str::FromStr;
use std::task::Poll;
use std::time::Duration;
use tracing_subscriber::EnvFilter;
use zeroize::Zeroizing;

mod behaviour;
mod config;
mod http_service;

const BOOTSTRAP_INTERVAL: Duration = Duration::from_secs(5 * 60);

#[derive(Debug, Parser)]
#[clap(name = "libp2p server", about = "A rust-libp2p server binary.")]
struct Opts {
Expand Down Expand Up @@ -125,17 +120,8 @@ async fn main() -> Result<(), Box<dyn Error>> {
}
});

let mut bootstrap_timer = Delay::new(BOOTSTRAP_INTERVAL);

loop {
if let Poll::Ready(()) = futures::poll!(&mut bootstrap_timer) {
bootstrap_timer.reset(BOOTSTRAP_INTERVAL);
let _ = swarm
.behaviour_mut()
.kademlia
.as_mut()
.map(|k| k.bootstrap());
}
let _ = swarm.behaviour_mut().kademlia.as_mut().map(|k| k.poll());
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved

let event = swarm.next().await.expect("Swarm not to terminate.");
metrics.record(&event);
Expand Down
5 changes: 5 additions & 0 deletions protocols/kad/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
## 0.45.2
PanGan21 marked this conversation as resolved.
Show resolved Hide resolved

- Add `refresh_interval` config and `Behaviour::poll` function used to poll `Behaviour::boostrap`
PanGan21 marked this conversation as resolved.
Show resolved Hide resolved
See [PR 4838](https://github.com/libp2p/rust-libp2p/pull/4838)

## 0.45.1

- Fix a bug where calling `Behaviour::remove_address` with an address not in the peer's bucket would remove the peer from the routing table if the bucket has only one address left.
Expand Down
2 changes: 1 addition & 1 deletion protocols/kad/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name = "libp2p-kad"
edition = "2021"
rust-version = { workspace = true }
description = "Kademlia protocol for libp2p"
version = "0.45.1"
version = "0.45.2"
authors = ["Parity Technologies <admin@parity.io>"]
license = "MIT"
repository = "https://github.com/libp2p/rust-libp2p"
Expand Down
33 changes: 33 additions & 0 deletions protocols/kad/src/behaviour.rs
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ use crate::record::{
};
use crate::K_VALUE;
use fnv::{FnvHashMap, FnvHashSet};
use futures_timer::Delay;
use instant::Instant;
use libp2p_core::{ConnectedPoint, Endpoint, Multiaddr};
use libp2p_identity::PeerId;
Expand Down Expand Up @@ -116,6 +117,9 @@ pub struct Behaviour<TStore> {

/// The record storage.
store: TStore,

/// The interval used by [`Behaviour::poll`] to call [`Behaviour::bootstrap`].
refresh_interval: Option<Duration>,
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved
}

/// The configurable strategies for the insertion of peers
Expand Down Expand Up @@ -181,6 +185,7 @@ pub struct Config {
provider_publication_interval: Option<Duration>,
kbucket_inserts: BucketInserts,
caching: Caching,
refresh_interval: Option<Duration>,
}

impl Default for Config {
Expand All @@ -197,6 +202,7 @@ impl Default for Config {
provider_record_ttl: Some(Duration::from_secs(24 * 60 * 60)),
kbucket_inserts: BucketInserts::OnConnected,
caching: Caching::Enabled { max_peers: 1 },
refresh_interval: Some(Duration::from_secs(5 * 60)),
}
}
}
Expand Down Expand Up @@ -391,6 +397,14 @@ impl Config {
self.caching = c;
self
}

/// Sets the interval on which [`Behaviour::bootstrap`] is called from [`Behaviour::poll`]
///
/// `None` means that [`Behaviour::bootstrap`] is not called from [`Behaviour::poll`]
PanGan21 marked this conversation as resolved.
Show resolved Hide resolved
pub fn set_refresh_interval(&mut self, interval: Option<Duration>) -> &mut Self {
self.refresh_interval = interval;
self
}
}

impl<TStore> Behaviour<TStore>
Expand Down Expand Up @@ -448,6 +462,7 @@ where
mode: Mode::Client,
auto_mode: true,
no_events_waker: None,
refresh_interval: config.refresh_interval,
}
}

Expand Down Expand Up @@ -1005,6 +1020,24 @@ where
}
}

/// Asynchronously polls the Kademlia behavior, triggering [`Behaviour::bootstrap`] if necessary.
///
/// This function checks the refresh interval and, if ready, resets the timer and
/// triggers the bootstrap operation. It returns a `Result<(), NoKnownPeers>` where
/// Ok(()) indicates success, and Err(NoKnownPeers) is returned if there are no known peers
/// during the bootstrap operation. See [`Behaviour::bootstrap`] for more details.
pub async fn poll(&mut self) -> Result<(), NoKnownPeers> {
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved
if let Some(refresh_interval) = &mut self.refresh_interval {
let mut bootstrap_timer = Delay::new(*refresh_interval);
if let Poll::Ready(()) = futures::poll!(&mut bootstrap_timer) {
bootstrap_timer.reset(*refresh_interval);
self.bootstrap()?;
};
}

Ok(())
}

fn reconfigure_mode(&mut self) {
if self.connections.is_empty() {
return;
Expand Down