Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use libp2p in catalyst #1245

Merged
merged 66 commits into from
Jun 16, 2020
Merged
Show file tree
Hide file tree
Changes from 60 commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
64a30da
Start of libp2p intergration in catalyst
May 7, 2020
2d13dc7
Refactor LibP2P use interfaces and more DI
May 7, 2020
40456fe
Update PoaNode and Tests
May 12, 2020
84ce778
Update ip/port to use multiaddress in more places
May 13, 2020
a3aa94d
Update unit tests
May 14, 2020
6425ec0
Update more unit tests
May 14, 2020
7711cbd
Add multi address in cli
May 15, 2020
8356510
Update proto submodule
May 15, 2020
18e964e
Update PoaConsensus intergration
May 16, 2020
db7dd4e
Update POA Concensus
May 18, 2020
73a1be3
Update get peers by repo
May 18, 2020
4ff65c0
Rename PeerId to Address, Add more PeerRepo signatures, Find POA node…
May 21, 2020
fac55e8
Get POA nodes by PublicKey
May 21, 2020
5873632
Use string instead of bytearray for publicKey
May 22, 2020
0c5988f
Create custom libp2p protocol
May 27, 2020
2d753db
Backup before updating protobuf ver
May 27, 2020
dfc6663
Use catalyst-protocol libp2p in peer client for testing
May 27, 2020
1d426c6
Update minimum peers
May 27, 2020
58679ad
Use multi-address instead of peerid libp2p
May 27, 2020
4ce16f7
Add try catch to peer client
May 27, 2020
dc48428
Use peer client broadcast method
May 28, 2020
f723bbe
Remove Dto
May 29, 2020
1b57c69
Revert "Remove Dto"
May 29, 2020
9f51153
Remove Dto
May 29, 2020
4e52bb1
Fix sync tests
May 29, 2020
0a24966
Update sync
May 31, 2020
b37f621
Add cancellation tokens and better async
May 31, 2020
4e5d8eb
Fix POA peer repository get peer by public key
May 31, 2020
fed140f
Blacklist peers that cannot be connected to but remove the black list…
May 31, 2020
da5654b
Update Catalyst LibP2P protocol intergration tests
May 31, 2020
e82abba
Add handlers to libp2p networking
Jun 1, 2020
27e7f4e
Add handlers to broadcast
Jun 1, 2020
c186701
Update peer client
Jun 1, 2020
b50a515
Fix bug with sending signed message
Jun 1, 2020
a2ba987
Update POA nodes config
Jun 1, 2020
6100459
Remove RPC Server and Client from Project, as EthController has been …
Jun 1, 2020
f13e4f0
Remove correlation handler from pubsub
Jun 1, 2020
9a65fc6
Comment out test to test pipeline
Jun 2, 2020
525bb2a
Add trait to CatalystProtocol intergration tests
Jun 2, 2020
64e131c
Update protocol and refactor
Jun 2, 2020
d34da16
Comment out unit tests
Jun 2, 2020
0a82208
Add rpc projects back to sln
Jun 2, 2020
09c49d7
Peer info request
Jun 2, 2020
459abd3
Update rpc unit tests
Jun 2, 2020
5dc5b4c
Comment out unit test for verify rpc
Jun 2, 2020
005c684
Comment out unit test for sign rpc
Jun 2, 2020
0940dba
Remove channel handler in rpc
Jun 3, 2020
bcfd8fe
Update ports for Catalyst libp2p protocol
Jun 3, 2020
0c32838
Uncomment some todo tests and update PeerId handler to use MultiAddress
Jun 3, 2020
c15740f
Fix FavouriteByHashAndVoterComparerTests
Jun 3, 2020
c4a85c6
Variable name refactor
Jun 3, 2020
9cd4585
Remove inmemory sharp repo from Catalyst.Abstraction packages
Jun 3, 2020
da1a252
Update protocol submodule
Jun 3, 2020
0356731
Revert ignore attribute on IntergrationTest for 1241
Jun 4, 2020
0f2c2fb
Remove unused DI from PeerService
Jun 4, 2020
728b9a9
Refactor catalyst node
Jun 4, 2020
fc1698c
Use peer client instead of broadcast manager on TransactionReceivedEvent
Jun 8, 2020
329fd48
Merge remote-tracking branch 'origin/develop' into use-libp2p-in-cata…
Jun 9, 2020
4051f41
Remove submodule ffi
Jun 9, 2020
a0f69fa
Uncomment depricated tests
Jun 10, 2020
3fc7de2
Refactor some codacy issues
Jun 10, 2020
5ebea62
Refactor for codacy
Jun 11, 2020
fe8749c
Refactor more issues for codacy
Jun 12, 2020
4699ffd
Refactor for codacy
Jun 12, 2020
57be0d4
Refactor for codacy
Jun 12, 2020
b7e46e6
Update for some autofac issues
Jun 15, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
3 changes: 0 additions & 3 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,6 @@
[submodule "submodules/Protocol"]
path = submodules/Protocol
url = https://github.com/catalyst-network/Protocol.git
[submodule "submodules/Cryptography.FFI.Rust"]
path = submodules/Cryptography.FFI.Rust
url = https://github.com/catalyst-network/Cryptography.FFI.Rust.git
[submodule "docs/articles"]
path = docs/articles
url = https://github.com/catalyst-network/Catalyst.Framework.wiki.git
3 changes: 2 additions & 1 deletion src/Catalyst.Abstractions/Cli/Commands/ICommandContext.cs
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
using Catalyst.Abstractions.IO.Transport;
using Catalyst.Abstractions.Rpc;
using Catalyst.Protocol.Peer;
using MultiFormats;

namespace Catalyst.Abstractions.Cli.Commands
{
Expand All @@ -48,7 +49,7 @@ public interface ICommandContext

/// <summary>Gets the peer identifier.</summary>
/// <value>The peer identifier.</value>
PeerId PeerId { get; }
MultiAddress Address { get; }

/// <summary>Gets the connected node.</summary>
/// <param name="nodeId">The node identifier located in configuration.</param>
Expand Down
14 changes: 3 additions & 11 deletions src/Catalyst.Abstractions/Cli/Options/IGetPeerInfoOptions.cs
Original file line number Diff line number Diff line change
Expand Up @@ -34,19 +34,11 @@ public interface IGetPeerInfoOptions
string Node { get; set; }

/// <summary>
/// Gets or sets the ip address.
/// Gets or sets the address.
/// </summary>
/// <value>
/// The ip address.
/// The multi address.
/// </value>
string IpAddress { get; set; }

/// <summary>
/// Gets or sets the public key.
/// </summary>
/// <value>
/// The public key.
/// </value>
string PublicKey { get; set; }
public string Address { get; set; }
}
}
14 changes: 3 additions & 11 deletions src/Catalyst.Abstractions/Cli/Options/IPeerBlackListingOptions.cs
Original file line number Diff line number Diff line change
Expand Up @@ -35,19 +35,11 @@ public interface IPeerBlackListingOptions : IOptionsBase
bool BlackListFlag { get; set; }

/// <summary>
/// Gets or sets the ip address.
/// Gets or sets the address.
/// </summary>
/// <value>
/// The ip address.
/// The multi address.
/// </value>
string IpAddress { get; set; }

/// <summary>
/// Gets or sets the public key.
/// </summary>
/// <value>
/// The public key.
/// </value>
string PublicKey { get; set; }
Xela101 marked this conversation as resolved.
Show resolved Hide resolved
public string Address { get; set; }
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,6 @@ namespace Catalyst.Abstractions.Cli.Options
{
public interface IPeerReputationOptions : IOptionsBase
{
string IpAddress { get; set; }

string PublicKey { get; set; }
public string Address { get; set; }
}
}
10 changes: 3 additions & 7 deletions src/Catalyst.Abstractions/Cli/Options/IRemovePeerOptions.cs
Original file line number Diff line number Diff line change
Expand Up @@ -25,12 +25,8 @@ namespace Catalyst.Abstractions.Cli.Options
{
public interface IRemovePeerOptions : IOptionsBase
{
/// <summary>Gets or sets the public key.</summary>
/// <value>The public key.</value>
string PublicKey { get; set; }

/// <summary>Gets or sets the ip.</summary>
/// <value>The ip.</value>
string Ip { get; set; }
/// <summary>Gets or sets the address.</summary>
/// <value>The multi address.</value>
public string Address { get; set; }
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ public interface IBitswapService : IService
/// <summary>
/// Provides access to other peers.
/// </summary>
SwarmService SwarmService { get; set; }
ISwarmService SwarmService { get; set; }

/// <summary>
/// Provides access to blocks of data.
Expand Down
8 changes: 4 additions & 4 deletions src/Catalyst.Abstractions/Dfs/IDfsService.cs
Original file line number Diff line number Diff line change
Expand Up @@ -61,21 +61,21 @@ public interface IDfsService : ICoreApi, IService, IDisposable
/// <summary>
/// Manages communication with other peers.
/// </summary>
SwarmService SwarmService { get; }
ISwarmService SwarmService { get; }

/// <summary>
/// Manages publishng and subscribing to messages.
/// </summary>
PubSubService PubSubService { get; }
IPubSubService PubSubService { get; }

/// <summary>
/// Exchange blocks with other peers.
/// </summary>
IBitswapService BitSwapService { get; }

/// <summary>
/// Finds information with a distributed hash table.
/// </summary>
DhtService DhtService { get; }
IDhtService DhtService { get; }
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
using Catalyst.Abstractions.IO.Messaging.Correlation;
using Catalyst.Protocol.Peer;
using DotNetty.Transport.Channels;
using MultiFormats;

namespace Catalyst.Abstractions.FileTransfer
{
Expand Down Expand Up @@ -81,11 +82,11 @@ public interface IFileTransferInformation : IDisposable

/// <summary>Gets or sets the recipient identifier.</summary>
/// <value>The recipient identifier.</value>
PeerId RecipientId { get; set; }
MultiAddress Recipient { get; set; }

/// <summary>Gets or sets the peer identifier.</summary>
/// <value>The peer identifier.</value>
PeerId PeerId { get; set; }
MultiAddress Address { get; set; }

/// <summary>The cancellation token</summary>
CancellationToken CancellationToken { get; set; }
Expand Down
33 changes: 33 additions & 0 deletions src/Catalyst.Abstractions/Hashing/HashExtensions.cs
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@
#endregion

using System.Buffers;
using System.Text;
using DotNetty.Common.Utilities;
using Google.Protobuf;
using MultiFormats;

Expand Down Expand Up @@ -62,6 +64,37 @@ public static MultiHash ComputeMultiHash(this IHashProvider provider, IMessage m
}
}

public static MultiHash ComputeMultiHash(this IHashProvider provider, string message, byte[] suffix)
{
return ComputeMultiHash(provider, Encoding.UTF8.GetBytes(message), suffix);
}

public static MultiHash ComputeMultiHash(this IHashProvider provider, byte[] message, byte[] suffix)
{
var calculateSize = message.Length;

var required = calculateSize + suffix.Length;
var array = ArrayPool<byte>.Shared.Rent(required);

try
{
message.CopyTo(array, 0);
suffix.CopyTo(array, calculateSize);

var result = provider.ComputeMultiHash(array, 0, required);
return result;
}
finally
{
ArrayPool<byte>.Shared.Return(array);
}
}

public static MultiHash ComputeMultiHash(this IHashProvider provider, MultiAddress address, byte[] suffix)
{
return ComputeMultiHash(provider, address.ToArray(), suffix);
}

/// <summary>
/// Serializes the <paramref name="message"/> returns the hash of it.
/// </summary>
Expand Down
27 changes: 27 additions & 0 deletions src/Catalyst.Abstractions/IO/Handlers/IInboundMessageHandler.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
#region LICENSE

/**
* Copyright (c) 2019 Catalyst Network
*
* This file is part of Catalyst.Node <https://github.com/catalyst-network/Catalyst.Node>
*
* Catalyst.Node is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 2 of the License, or
* (at your option) any later version.
*
* Catalyst.Node is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Catalyst.Node. If not, see <https://www.gnu.org/licenses/>.
*/

#endregion

namespace Catalyst.Abstractions.IO.Handlers
{
public interface IInboundMessageHandler : IMessageHandler { }
}
33 changes: 33 additions & 0 deletions src/Catalyst.Abstractions/IO/Handlers/IMessageHandler.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
#region LICENSE

/**
* Copyright (c) 2019 Catalyst Network
*
* This file is part of Catalyst.Node <https://github.com/catalyst-network/Catalyst.Node>
*
* Catalyst.Node is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 2 of the License, or
* (at your option) any later version.
*
* Catalyst.Node is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Catalyst.Node. If not, see <https://www.gnu.org/licenses/>.
*/

#endregion

using Catalyst.Protocol.Wire;
using System.Threading.Tasks;

namespace Catalyst.Abstractions.IO.Handlers
{
public interface IMessageHandler
{
Task<bool> ProcessAsync(ProtocolMessage message);
}
}
27 changes: 27 additions & 0 deletions src/Catalyst.Abstractions/IO/Handlers/IOutboundMessageHandler.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
#region LICENSE

/**
* Copyright (c) 2019 Catalyst Network
*
* This file is part of Catalyst.Node <https://github.com/catalyst-network/Catalyst.Node>
*
* Catalyst.Node is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 2 of the License, or
* (at your option) any later version.
*
* Catalyst.Node is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Catalyst.Node. If not, see <https://www.gnu.org/licenses/>.
*/

#endregion

namespace Catalyst.Abstractions.IO.Handlers
{
public interface IOutboundMessageHandler : IMessageHandler { }
}
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,13 @@

using Catalyst.Protocol.Peer;
using Google.Protobuf;
using MultiFormats;

namespace Catalyst.Abstractions.IO.Messaging.Correlation
{
public interface ICacheEvictionEvent<out T> where T : IMessage
{
T EvictedContent { get; }
PeerId PeerId { get; }
MultiAddress Address { get; }
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -24,13 +24,14 @@
using System;
using Catalyst.Protocol.Peer;
using Google.Protobuf;
using MultiFormats;

namespace Catalyst.Abstractions.IO.Messaging.Correlation
{
public interface ICorrelatableMessage<T> where T : IMessage
{
T Content { get; set; }
PeerId Recipient { get; set; }
MultiAddress Recipient { get; set; }
DateTimeOffset SentAt { get; set; }
}
}
6 changes: 3 additions & 3 deletions src/Catalyst.Abstractions/IO/Messaging/Dto/IMessageDto.cs
Original file line number Diff line number Diff line change
Expand Up @@ -22,16 +22,16 @@
#endregion

using Catalyst.Abstractions.IO.Messaging.Correlation;
using Catalyst.Protocol.Peer;
using DotNetty.Transport.Channels;
using Google.Protobuf;
using MultiFormats;

namespace Catalyst.Abstractions.IO.Messaging.Dto
{
public interface IMessageDto<out T> : IAddressedEnvelope<T> where T : IMessage
{
ICorrelationId CorrelationId { get; }
PeerId RecipientPeerIdentifier { get; }
PeerId SenderPeerIdentifier { get; }
MultiAddress RecipientAddress { get; }
MultiAddress SenderAddress { get; }
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,12 @@
using Catalyst.Protocol.Peer;
using DotNetty.Transport.Channels;
using Google.Protobuf;
using MultiFormats;

namespace Catalyst.Abstractions.IO.Observers
{
public interface IRpcResponseObserver : IResponseMessageObserver
{
void HandleResponseObserver(IMessage messageDto, IChannelHandlerContext channelHandlerContext, PeerId senderPeerIdentifier, ICorrelationId correlationId);
void HandleResponseObserver(IMessage messageDto, IChannelHandlerContext channelHandlerContext, MultiAddress senderentifier, ICorrelationId correlationId);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,14 @@
using System.Security.Cryptography.X509Certificates;
using System.Threading.Tasks;
using Catalyst.Abstractions.IO.EventLoop;
using MultiFormats;

namespace Catalyst.Abstractions.IO.Transport.Channels
{
public interface IChannelFactory
{
Task<IObservableChannel> BuildChannelAsync(IEventLoopGroupFactory eventLoopGroupFactory,
IPAddress targetAddress,
int targetPort,
MultiAddress address,
X509Certificate2 certificate = null);
}
}
3 changes: 1 addition & 2 deletions src/Catalyst.Abstractions/Keystore/IKeyApi.cs
Original file line number Diff line number Diff line change
Expand Up @@ -131,9 +131,8 @@ Task<IKey> CreateAsync(string name,
Task<IKey> ImportAsync(string name, string pem, char[] password, CancellationToken cancel = default);

Task<IKey> GetKeyAsync(string self);
Task<string> GetIpfsPublicKeyAsync(string name, CancellationToken cancel = default);
Task<string> GetDfsPublicKeyAsync(string name, CancellationToken cancel = default);

Task<AsymmetricKeyParameter> GetPublicKeyAsync(string self);
Task<AsymmetricKeyParameter> GetPrivateKeyAsync(string self);

Task<byte[]> CreateProtectedDataAsync(string keyName,
Expand Down
Loading