Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove libp2p dependency from sc-network-sync #4974

Merged
merged 61 commits into from
Sep 18, 2024
Merged
Show file tree
Hide file tree
Changes from 55 commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
51f9a34
Removed libp2p dependancy from sc_network_sync
ndkazu Jul 8, 2024
b4e3f5a
Preparing an alternative to libp2p dependant OutBoundFailure
ndkazu Jul 9, 2024
fa242e5
Merge branch 'master' into libp2p
ndkazu Jul 9, 2024
6a1fcf1
Agnostic sc-network-sync
ndkazu Jul 10, 2024
b20e332
Merge branch 'paritytech:master' into libp2p
ndkazu Jul 10, 2024
fdac723
Merge branch 'master' into libp2p
ndkazu Jul 15, 2024
b33728b
Update substrate/client/network/src/request_responses.rs
ndkazu Jul 16, 2024
d8fbc30
Remove crate libp2p from sc-authority-discovery
ndkazu Jul 16, 2024
86d50af
Merge branch 'libp2p' of github.com:ndkazu/polkadot-sdk into libp2p
ndkazu Jul 16, 2024
2f09ebd
Removed event
ndkazu Jul 17, 2024
d0e2084
Add a custom InboundFailure
ndkazu Jul 17, 2024
3f7e5ad
Removed the variant Network
ndkazu Jul 18, 2024
6dd1bd3
Changed the type of
ndkazu Jul 20, 2024
97a5453
Merge branch 'master' into libp2p
ndkazu Jul 20, 2024
12db7fa
Merge branch 'master' into libp2p
ndkazu Jul 22, 2024
02f0085
Merge branch 'paritytech:master' into libp2p
ndkazu Jul 24, 2024
372f9fc
Replaced Network2 by Network, and added comments to enums
ndkazu Jul 24, 2024
28830fe
Replaced Network2 by Network
ndkazu Jul 24, 2024
f02b17c
Removed Event0, Need to remove ToSwarm dependance to libp2p
ndkazu Jul 26, 2024
ccbc94a
Merge branch 'paritytech:master' into libp2p
ndkazu Aug 3, 2024
7229689
Merge branch 'master' into libp2p
ndkazu Sep 2, 2024
8609e59
Merge branch 'paritytech:master' into libp2p
ndkazu Sep 3, 2024
54c54c5
Ready for review
ndkazu Sep 3, 2024
ae006e1
First corrections
ndkazu Sep 5, 2024
a52c9d4
Second round of corrections
ndkazu Sep 5, 2024
73860e6
cargo +nightly fmt
ndkazu Sep 5, 2024
7ed1124
Merge branch 'master' into libp2p
ndkazu Sep 6, 2024
db03bc5
Revert "cargo +nightly fmt"
ndkazu Sep 7, 2024
3dd3834
Merge branch 'master' into libp2p
ndkazu Sep 7, 2024
f194e5f
".git/.scripts/commands/fmt/fmt.sh"
Sep 9, 2024
6ac72b3
Removed all #[error(dial-failure)]
ndkazu Sep 9, 2024
104b1c4
Merge branch 'master' into libp2p
ndkazu Sep 9, 2024
e056e0e
Merge branch 'master' into libp2p
ndkazu Sep 11, 2024
53ea246
Replaced CustomOutboundFailure by OutboundFailure
ndkazu Sep 12, 2024
65c732e
Merge branch 'master' into libp2p
ndkazu Sep 12, 2024
da7adc3
first round of corrections
ndkazu Sep 13, 2024
7c916bc
Merge branch 'master' into libp2p
ndkazu Sep 13, 2024
aab8614
Reverted where it was necessary
ndkazu Sep 13, 2024
812734c
corrections
ndkazu Sep 13, 2024
35e884e
Merge branch 'master' into libp2p
ndkazu Sep 13, 2024
0062a74
Merge branch 'master' into libp2p
ndkazu Sep 13, 2024
dd08c06
Merge branch 'master' into libp2p
ndkazu Sep 13, 2024
cc0c400
Merge branch 'master' into libp2p
ndkazu Sep 14, 2024
67fe210
Created a prdoc
ndkazu Sep 16, 2024
c608293
Merge branch 'master' into libp2p
ndkazu Sep 16, 2024
a74ebd0
Merge branch 'master' into libp2p
ndkazu Sep 17, 2024
73d4005
implemented From<...> for OutbounFailure & InboundFailure
ndkazu Sep 17, 2024
2fcf160
Reverted changes unrelated to the purpose of the PR
ndkazu Sep 17, 2024
c2abf70
more revert
ndkazu Sep 17, 2024
9c8248b
Merge branch 'master' into libp2p
ndkazu Sep 17, 2024
34cdc57
".git/.scripts/commands/fmt/fmt.sh"
Sep 17, 2024
1ee40e1
Added newline at EOF
ndkazu Sep 17, 2024
a354c2e
Further corrections
ndkazu Sep 17, 2024
abc6f68
correcting format
ndkazu Sep 17, 2024
dfcc689
more format correction
ndkazu Sep 17, 2024
7bb250f
removed space
ndkazu Sep 17, 2024
ba729b5
".git/.scripts/commands/fmt/fmt.sh"
Sep 17, 2024
c969d26
Latest review corrections
ndkazu Sep 17, 2024
e226706
Manual Formatting
ndkazu Sep 17, 2024
f1e0164
Correct newline at EOF in prdoc
dmitry-markin Sep 18, 2024
27c8e13
Merge branch 'master' into libp2p
lexnv Sep 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

16 changes: 16 additions & 0 deletions prdoc/pr_4974.prdoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Schema: Polkadot SDK PRDoc Schema (prdoc) v1.0.0
# See doc at https://raw.githubusercontent.com/paritytech/polkadot-sdk/master/prdoc/schema_user.json

title: "Remove libp2p dependency from sc-network-sync"

doc:
- audience: Node Dev
description: |
This PR removes `libp2p::request_response::OutboundFailure` from `substrate/client/network/sync/src/engine.rs`.

crates:
- name: sc-network
bump: patch
- name: sc-network-sync
bump: patch

109 changes: 101 additions & 8 deletions substrate/client/network/src/request_responses.rs
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ use crate::{
use futures::{channel::oneshot, prelude::*};
use libp2p::{
core::{Endpoint, Multiaddr},
request_response::{self, Behaviour, Codec, Message, ProtocolSupport, ResponseChannel},
request_response::{self, Behaviour, Codec, ProtocolSupport, ResponseChannel},
swarm::{
behaviour::{ConnectionClosed, FromSwarm},
handler::multi::MultiHandler,
Expand All @@ -56,6 +56,8 @@ use libp2p::{

use std::{
collections::{hash_map::Entry, HashMap},
fmt,
fmt::{Display, Formatter},
io, iter,
ops::Deref,
pin::Pin,
Expand All @@ -64,7 +66,93 @@ use std::{
time::{Duration, Instant},
};

pub use libp2p::request_response::{Config, InboundFailure, OutboundFailure, RequestId};
pub use libp2p::request_response::{Config, RequestId};

/// Possible failures occurring in the context of sending an outbound request and receiving the
/// response.

ndkazu marked this conversation as resolved.
Show resolved Hide resolved
#[derive(Debug, thiserror::Error)]
pub enum OutboundFailure {
/// The request could not be sent because a dialing attempt failed.
DialFailure,
/// The request timed out before a response was received.
Timeout,
ndkazu marked this conversation as resolved.
Show resolved Hide resolved
/// The connection closed before a response was received.
ConnectionClosed,
/// The remote supports none of the requested protocols.
UnsupportedProtocols,
}

impl Display for OutboundFailure {
fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
match self {
OutboundFailure::DialFailure => write!(f, "Failed to dial the requested peer"),
ndkazu marked this conversation as resolved.
Show resolved Hide resolved
OutboundFailure::Timeout => write!(f, "Timeout while waiting for a response"),
OutboundFailure::ConnectionClosed =>
write!(f, "Connection was closed before a response was received"),
OutboundFailure::UnsupportedProtocols =>
write!(f, "The remote supports none of the requested protocols"),
}
}
}

impl From<request_response::OutboundFailure> for OutboundFailure {
fn from(out: request_response::OutboundFailure) -> Self {
match out {
request_response::OutboundFailure::DialFailure => OutboundFailure::DialFailure,
request_response::OutboundFailure::Timeout => OutboundFailure::Timeout,
request_response::OutboundFailure::ConnectionClosed =>
OutboundFailure::ConnectionClosed,
request_response::OutboundFailure::UnsupportedProtocols =>
OutboundFailure::UnsupportedProtocols,
}
}
}

/// Possible failures occurring in the context of receiving an inbound request and sending a
/// response.

ndkazu marked this conversation as resolved.
Show resolved Hide resolved
#[derive(Debug, thiserror::Error)]
ndkazu marked this conversation as resolved.
Show resolved Hide resolved
pub enum InboundFailure {
/// The inbound request timed out, either while reading the incoming request or before a
/// response is sent
Timeout,
/// The connection closed before a response could be send.
ConnectionClosed,
/// The local peer supports none of the protocols requested by the remote.
UnsupportedProtocols,
/// The local peer failed to respond to an inbound request
ResponseOmission,
}

ndkazu marked this conversation as resolved.
Show resolved Hide resolved
impl Display for InboundFailure {
fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
match self {
InboundFailure::Timeout =>
write!(f, "Timeout while receiving request or sending response"),
InboundFailure::ConnectionClosed =>
write!(f, "Connection was closed before a response could be sent"),
InboundFailure::UnsupportedProtocols =>
write!(f, "The local peer supports none of the protocols requested by the remote"),
InboundFailure::ResponseOmission => write!(
f,
"The response channel was dropped without sending a response to the remote"
),
}
}
}

impl From<request_response::InboundFailure> for InboundFailure {
fn from(out: request_response::InboundFailure) -> Self {
match out {
request_response::InboundFailure::ResponseOmission => InboundFailure::ResponseOmission,
request_response::InboundFailure::Timeout => InboundFailure::Timeout,
request_response::InboundFailure::ConnectionClosed => InboundFailure::ConnectionClosed,
request_response::InboundFailure::UnsupportedProtocols =>
InboundFailure::UnsupportedProtocols,
}
}
}

ndkazu marked this conversation as resolved.
Show resolved Hide resolved
/// Error in a request.
#[derive(Debug, thiserror::Error)]
Expand Down Expand Up @@ -684,7 +772,10 @@ impl NetworkBehaviour for RequestResponsesBehaviour {
// Received a request from a remote.
request_response::Event::Message {
peer,
message: Message::Request { request_id, request, channel, .. },
message:
request_response::Message::Request {
ndkazu marked this conversation as resolved.
Show resolved Hide resolved
request_id, request, channel, ..
},
} => {
self.pending_responses_arrival_time
.insert((protocol.clone(), request_id).into(), Instant::now());
Expand Down Expand Up @@ -746,7 +837,7 @@ impl NetworkBehaviour for RequestResponsesBehaviour {
// Received a response from a remote to one of our requests.
request_response::Event::Message {
peer,
message: Message::Response { request_id, response },
message: request_response::Message::Response { request_id, response },
ndkazu marked this conversation as resolved.
Show resolved Hide resolved
..
} => {
let (started, delivered) = match self
Expand Down Expand Up @@ -808,7 +899,9 @@ impl NetworkBehaviour for RequestResponsesBehaviour {
}) => {
// Try using the fallback request if the protocol was not
// supported.
if let OutboundFailure::UnsupportedProtocols = error {
if let request_response::OutboundFailure::UnsupportedProtocols =
error
{
if let Some((fallback_request, fallback_protocol)) =
fallback_request
{
Expand All @@ -829,7 +922,7 @@ impl NetworkBehaviour for RequestResponsesBehaviour {
}

if response_tx
.send(Err(RequestFailure::Network(error.clone())))
.send(Err(RequestFailure::Network(error.clone().into())))
.is_err()
{
log::debug!(
Expand All @@ -856,7 +949,7 @@ impl NetworkBehaviour for RequestResponsesBehaviour {
peer,
protocol: protocol.clone(),
duration: started.elapsed(),
result: Err(RequestFailure::Network(error)),
result: Err(RequestFailure::Network(error.into())),
};

return Poll::Ready(ToSwarm::GenerateEvent(out))
Expand All @@ -873,7 +966,7 @@ impl NetworkBehaviour for RequestResponsesBehaviour {
let out = Event::InboundRequest {
peer,
protocol: protocol.clone(),
result: Err(ResponseFailure::Network(error)),
result: Err(ResponseFailure::Network(error.into())),
};
return Poll::Ready(ToSwarm::GenerateEvent(out))
},
Expand Down
1 change: 0 additions & 1 deletion substrate/client/network/sync/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ async-trait = { workspace = true }
codec = { features = ["derive"], workspace = true, default-features = true }
futures = { workspace = true }
futures-timer = { workspace = true }
libp2p = { workspace = true }
log = { workspace = true, default-features = true }
mockall = { workspace = true }
prost = { workspace = true }
Expand Down
3 changes: 1 addition & 2 deletions substrate/client/network/sync/src/engine.rs
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,6 @@ use crate::{

use codec::{Decode, DecodeAll, Encode};
use futures::{channel::oneshot, FutureExt, StreamExt};
use libp2p::request_response::OutboundFailure;
use log::{debug, error, trace, warn};
use prometheus_endpoint::{
register, Counter, Gauge, MetricSource, Opts, PrometheusError, Registry, SourcedGauge, U64,
Expand All @@ -56,7 +55,7 @@ use sc_consensus::{import_queue::ImportQueueService, IncomingBlock};
use sc_network::{
config::{FullNetworkConfiguration, NotificationHandshake, ProtocolId, SetConfig},
peer_store::PeerStoreProvider,
request_responses::{IfDisconnected, RequestFailure},
request_responses::{IfDisconnected, OutboundFailure, RequestFailure},
service::{
traits::{Direction, NotificationConfig, NotificationEvent, ValidationResult},
NotificationMetrics,
Expand Down
Loading