Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove libp2p dependency from sc-network-sync #4974

Merged
merged 61 commits into from
Sep 18, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
51f9a34
Removed libp2p dependancy from sc_network_sync
ndkazu Jul 8, 2024
b4e3f5a
Preparing an alternative to libp2p dependant OutBoundFailure
ndkazu Jul 9, 2024
fa242e5
Merge branch 'master' into libp2p
ndkazu Jul 9, 2024
6a1fcf1
Agnostic sc-network-sync
ndkazu Jul 10, 2024
b20e332
Merge branch 'paritytech:master' into libp2p
ndkazu Jul 10, 2024
fdac723
Merge branch 'master' into libp2p
ndkazu Jul 15, 2024
b33728b
Update substrate/client/network/src/request_responses.rs
ndkazu Jul 16, 2024
d8fbc30
Remove crate libp2p from sc-authority-discovery
ndkazu Jul 16, 2024
86d50af
Merge branch 'libp2p' of github.com:ndkazu/polkadot-sdk into libp2p
ndkazu Jul 16, 2024
2f09ebd
Removed event
ndkazu Jul 17, 2024
d0e2084
Add a custom InboundFailure
ndkazu Jul 17, 2024
3f7e5ad
Removed the variant Network
ndkazu Jul 18, 2024
6dd1bd3
Changed the type of
ndkazu Jul 20, 2024
97a5453
Merge branch 'master' into libp2p
ndkazu Jul 20, 2024
12db7fa
Merge branch 'master' into libp2p
ndkazu Jul 22, 2024
02f0085
Merge branch 'paritytech:master' into libp2p
ndkazu Jul 24, 2024
372f9fc
Replaced Network2 by Network, and added comments to enums
ndkazu Jul 24, 2024
28830fe
Replaced Network2 by Network
ndkazu Jul 24, 2024
f02b17c
Removed Event0, Need to remove ToSwarm dependance to libp2p
ndkazu Jul 26, 2024
ccbc94a
Merge branch 'paritytech:master' into libp2p
ndkazu Aug 3, 2024
7229689
Merge branch 'master' into libp2p
ndkazu Sep 2, 2024
8609e59
Merge branch 'paritytech:master' into libp2p
ndkazu Sep 3, 2024
54c54c5
Ready for review
ndkazu Sep 3, 2024
ae006e1
First corrections
ndkazu Sep 5, 2024
a52c9d4
Second round of corrections
ndkazu Sep 5, 2024
73860e6
cargo +nightly fmt
ndkazu Sep 5, 2024
7ed1124
Merge branch 'master' into libp2p
ndkazu Sep 6, 2024
db03bc5
Revert "cargo +nightly fmt"
ndkazu Sep 7, 2024
3dd3834
Merge branch 'master' into libp2p
ndkazu Sep 7, 2024
f194e5f
".git/.scripts/commands/fmt/fmt.sh"
Sep 9, 2024
6ac72b3
Removed all #[error(dial-failure)]
ndkazu Sep 9, 2024
104b1c4
Merge branch 'master' into libp2p
ndkazu Sep 9, 2024
e056e0e
Merge branch 'master' into libp2p
ndkazu Sep 11, 2024
53ea246
Replaced CustomOutboundFailure by OutboundFailure
ndkazu Sep 12, 2024
65c732e
Merge branch 'master' into libp2p
ndkazu Sep 12, 2024
da7adc3
first round of corrections
ndkazu Sep 13, 2024
7c916bc
Merge branch 'master' into libp2p
ndkazu Sep 13, 2024
aab8614
Reverted where it was necessary
ndkazu Sep 13, 2024
812734c
corrections
ndkazu Sep 13, 2024
35e884e
Merge branch 'master' into libp2p
ndkazu Sep 13, 2024
0062a74
Merge branch 'master' into libp2p
ndkazu Sep 13, 2024
dd08c06
Merge branch 'master' into libp2p
ndkazu Sep 13, 2024
cc0c400
Merge branch 'master' into libp2p
ndkazu Sep 14, 2024
67fe210
Created a prdoc
ndkazu Sep 16, 2024
c608293
Merge branch 'master' into libp2p
ndkazu Sep 16, 2024
a74ebd0
Merge branch 'master' into libp2p
ndkazu Sep 17, 2024
73d4005
implemented From<...> for OutbounFailure & InboundFailure
ndkazu Sep 17, 2024
2fcf160
Reverted changes unrelated to the purpose of the PR
ndkazu Sep 17, 2024
c2abf70
more revert
ndkazu Sep 17, 2024
9c8248b
Merge branch 'master' into libp2p
ndkazu Sep 17, 2024
34cdc57
".git/.scripts/commands/fmt/fmt.sh"
Sep 17, 2024
1ee40e1
Added newline at EOF
ndkazu Sep 17, 2024
a354c2e
Further corrections
ndkazu Sep 17, 2024
abc6f68
correcting format
ndkazu Sep 17, 2024
dfcc689
more format correction
ndkazu Sep 17, 2024
7bb250f
removed space
ndkazu Sep 17, 2024
ba729b5
".git/.scripts/commands/fmt/fmt.sh"
Sep 17, 2024
c969d26
Latest review corrections
ndkazu Sep 17, 2024
e226706
Manual Formatting
ndkazu Sep 17, 2024
f1e0164
Correct newline at EOF in prdoc
dmitry-markin Sep 18, 2024
27c8e13
Merge branch 'master' into libp2p
lexnv Sep 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

48 changes: 45 additions & 3 deletions substrate/client/network/src/request_responses.rs
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ use libp2p::{
ConnectionDenied, ConnectionId, NetworkBehaviour, PollParameters, THandler,
THandlerInEvent, THandlerOutEvent, ToSwarm,
},
PeerId,
PeerId,
ndkazu marked this conversation as resolved.
Show resolved Hide resolved
};

use std::{
Expand All @@ -62,14 +62,48 @@ use std::{
sync::Arc,
task::{Context, Poll},
time::{Duration, Instant},
fmt::{Display,Formatter},
fmt,
};

pub use libp2p::request_response::{Config, InboundFailure, OutboundFailure, RequestId};

ndkazu marked this conversation as resolved.
Show resolved Hide resolved
/// Adding a custom OutBoundFailure, not depending on libp2p
#[derive(Debug, thiserror::Error)]
#[allow(missing_docs)]
ndkazu marked this conversation as resolved.
Show resolved Hide resolved
ndkazu marked this conversation as resolved.
Show resolved Hide resolved
pub enum CustomOutboundFailure{
DialFailure,
Timeout,
ConnectionClosed,
UnsupportedProtocols,
ndkazu marked this conversation as resolved.
Show resolved Hide resolved
}

/// Implement Display trait
impl Display for CustomOutboundFailure{
fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result{
match self {
CustomOutboundFailure::DialFailure => write!(f, "DialFailure"),
CustomOutboundFailure::Timeout => write!(f, "Timeout"),
CustomOutboundFailure::ConnectionClosed => write!(f, "ConnectionClosed"),
CustomOutboundFailure::UnsupportedProtocols => write!(f, "UnsupportedProtocols"),
}
}
}

/// In preparation for a CustomOutBoundFailure Event
#[derive(Debug, Copy, Clone, PartialEq, Eq, Hash, PartialOrd, Ord)]
pub struct OutboundRequestId(u64);

ndkazu marked this conversation as resolved.
Show resolved Hide resolved
impl fmt::Display for OutboundRequestId {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "{}", self.0)
}
}

/// Error in a request.
#[derive(Debug, thiserror::Error)]
#[allow(missing_docs)]
pub enum RequestFailure {
pub enum RequestFailure{
#[error("We are not currently connected to the requested peer.")]
NotConnected,
#[error("Given protocol hasn't been registered.")]
Expand All @@ -80,6 +114,8 @@ pub enum RequestFailure {
Obsolete,
#[error("Problem on the network: {0}")]
Network(OutboundFailure),
#[error("Problem on the network: {0}")]
Network2(CustomOutboundFailure),
ndkazu marked this conversation as resolved.
Show resolved Hide resolved
ndkazu marked this conversation as resolved.
Show resolved Hide resolved
}

/// Configuration for a single request-response protocol.
Expand Down Expand Up @@ -248,6 +284,12 @@ pub enum Event {
/// Reputation changes.
changes: Vec<ReputationChange>,
},

/*CustomOutboundFailure {
peer: PeerId,
request_id: OutboundRequestId,
error: CustomOutboundFailure,
ndkazu marked this conversation as resolved.
Show resolved Hide resolved
}*/
}

/// Combination of a protocol name and a request id.
Expand Down Expand Up @@ -1734,4 +1776,4 @@ mod tests {
);
});
}
}
}
ndkazu marked this conversation as resolved.
Show resolved Hide resolved
11 changes: 11 additions & 0 deletions substrate/client/network/src/service.rs
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,8 @@ use sc_utils::mpsc::{tracing_unbounded, TracingUnboundedReceiver, TracingUnbound
use sp_runtime::traits::Block as BlockT;

pub use behaviour::{InboundFailure, OutboundFailure, ResponseFailure};
// Import our custom type
pub use crate::request_responses::CustomOutboundFailure;
pub use libp2p::identity::{DecodingError, Keypair, PublicKey};
pub use metrics::NotificationMetrics;
pub use protocol::NotificationsSink;
Expand Down Expand Up @@ -1530,6 +1532,7 @@ where
.with_label_values(&[&protocol])
.observe(duration.as_secs_f64());
},
// we also could remove Network cases here like we did in engine
ndkazu marked this conversation as resolved.
Show resolved Hide resolved
Err(err) => {
let reason = match err {
RequestFailure::NotConnected => "not-connected",
Expand All @@ -1543,6 +1546,14 @@ where
"connection-closed",
RequestFailure::Network(OutboundFailure::UnsupportedProtocols) =>
"unsupported",
RequestFailure::Network2(CustomOutboundFailure::DialFailure) =>
"dial-failure",
RequestFailure::Network2(CustomOutboundFailure::Timeout) => "timeout",
RequestFailure::Network2(CustomOutboundFailure::ConnectionClosed) =>
"connection-closed",
RequestFailure::Network2(CustomOutboundFailure::UnsupportedProtocols) =>
"unsupported",

};

metrics
Expand Down
1 change: 0 additions & 1 deletion substrate/client/network/sync/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ async-trait = { workspace = true }
codec = { features = ["derive"], workspace = true, default-features = true }
futures = { workspace = true }
futures-timer = { workspace = true }
libp2p = { workspace = true }
log = { workspace = true, default-features = true }
mockall = { workspace = true }
prost = { workspace = true }
Expand Down
14 changes: 8 additions & 6 deletions substrate/client/network/sync/src/engine.rs
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,6 @@ use futures::{
future::{BoxFuture, Fuse},
FutureExt, StreamExt,
};
use libp2p::request_response::OutboundFailure;
use log::{debug, error, trace, warn};
use prometheus_endpoint::{
register, Counter, Gauge, MetricSource, Opts, PrometheusError, Registry, SourcedGauge, U64,
Expand All @@ -61,7 +60,7 @@ use sc_consensus::{import_queue::ImportQueueService, IncomingBlock};
use sc_network::{
config::{FullNetworkConfiguration, NotificationHandshake, ProtocolId, SetConfig},
peer_store::PeerStoreProvider,
request_responses::{IfDisconnected, RequestFailure},
request_responses::{IfDisconnected, RequestFailure, CustomOutboundFailure},
service::{
traits::{Direction, NotificationConfig, NotificationEvent, ValidationResult},
NotificationMetrics,
Expand Down Expand Up @@ -1272,18 +1271,19 @@ where
Ok(Err(e)) => {
debug!(target: LOG_TARGET, "Request to peer {peer_id:?} failed: {e:?}.");

// Using Our custom type Network2(CustomOutboundFailure)
match e {
RequestFailure::Network(OutboundFailure::Timeout) => {
RequestFailure::Network2(CustomOutboundFailure::Timeout) => {
self.network_service.report_peer(peer_id, rep::TIMEOUT);
self.network_service
.disconnect_peer(peer_id, self.block_announce_protocol_name.clone());
},
RequestFailure::Network(OutboundFailure::UnsupportedProtocols) => {
RequestFailure::Network2(CustomOutboundFailure::UnsupportedProtocols) => {
self.network_service.report_peer(peer_id, rep::BAD_PROTOCOL);
self.network_service
.disconnect_peer(peer_id, self.block_announce_protocol_name.clone());
},
RequestFailure::Network(OutboundFailure::DialFailure) => {
RequestFailure::Network2(CustomOutboundFailure::DialFailure) => {
self.network_service
.disconnect_peer(peer_id, self.block_announce_protocol_name.clone());
},
Expand All @@ -1292,7 +1292,7 @@ where
self.network_service
.disconnect_peer(peer_id, self.block_announce_protocol_name.clone());
},
RequestFailure::Network(OutboundFailure::ConnectionClosed) |
RequestFailure::Network2(CustomOutboundFailure::ConnectionClosed) |
RequestFailure::NotConnected => {
self.network_service
.disconnect_peer(peer_id, self.block_announce_protocol_name.clone());
Expand All @@ -1307,6 +1307,8 @@ where
response receiver.",
);
},
//If OutboundFailure is used,
RequestFailure::Network(_) => {debug_assert!(false, "Don't use deprecated Network");},
dmitry-markin marked this conversation as resolved.
Show resolved Hide resolved
}
},
Err(oneshot::Canceled) => {
Expand Down