Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dot/rpc implement state_subscribeStorage RPC WebSocket method #983

Merged
merged 29 commits into from
Jul 16, 2020

Conversation

edwardmack
Copy link
Member

Changes

Tests

go test ./dot/rpc/...

OR connect a websocket client to node and send request:

{
    "jsonrpc": "2.0",
    "method": "state_subscribeStorage",
    "params": [],
    "id": 1
}

Checklist

  • I have read CODE_OF_CONDUCT and CONTRIBUTING
  • I have provided as much information as possible and necessary
  • I have reviewed my own pull request before requesting a review
  • All integration tests and required coverage checks are passing

Issues

@edwardmack
Copy link
Member Author

I see that code climate is complaining, let me know if you want me to refactor.

// TODO implement change storage trie so that block hash parameter works (See issue #834)
// SubscribeStorage Storage subscription. If storage keys are specified, it creates a message for each block which
// changes the specified storage keys. If none are specified, then it creates a message for every block.
// This endpoint communicates over the Websockt protocol, but this func should remain here so it's added to rpc_methods list
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

*websocket

// SubscribeStorage Storage subscription. If storage keys are specified, it creates a message for each block which
// changes the specified storage keys. If none are specified, then it creates a message for every block.
// This endpoint communicates over the Websockt protocol, but this func should remain here so it's added to rpc_methods list
func (sm *StateModule) SubscribeStorage(r *http.Request, req *StateStorageQueryRangeRequest, res *StorageChangeSetResponse) error {
Copy link
Contributor

@noot noot Jul 6, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also wondering, is there an unsubscribe method corresponding to the subscribe methods? if so, it would be nice to return the subscription ID

wss := &WebSocketSubscription{
WSConnection: conn,
SubscriptionType: subscriptionType,
Filter: filter,
}
wssub[sub] = wss
h.serverConfig.WSSubscriptions = wssub
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the WSSubscriptions should be stored in the HTTPServer struct (or in a separate wsServer struct) not in the config

if change != nil {
for i, sub := range h.serverConfig.WSSubscriptions {
if sub.SubscriptionType == SUB_STORAGE {
// check it change key is in subscription filter
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check if*

Key: key,
Value: value,
}
//fmt.Printf("SetStorage Key: %x Value: %x\n", key, value)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove?

Value: value,
}
//fmt.Printf("SetStorage Key: %x Value: %x\n", key, value)
// TODO ed add channel notify here
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this done?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should also be added to ClearStorage

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

Copy link
Contributor

@noot noot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall looking good, a few comments!

for change := range h.storageChan {

if change != nil {
for i, sub := range h.serverConfig.WSSubscriptions {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if it would be simpler to start a goroutine for each subscription, what do you think?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I've refactor websockets to better handle this.

@edwardmack
Copy link
Member Author

I've updated this, it's ready for another look. Let me know your thoughts.

@noot noot self-requested a review July 14, 2020 15:34
Copy link
Contributor

@noot noot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great work! i have a few smaller comments but overall looks good :)

dot/rpc/http.go Outdated
type WSConn struct {
wsconn *websocket.Conn
mu sync.Mutex
serverConfig *HTTPServerConfig
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this need the whole server config? or just the state interfaces?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No just BlockAPI and StorageAPI, I've updated.

dot/rpc/http.go Outdated
wsconn *websocket.Conn
mu sync.Mutex
serverConfig *HTTPServerConfig
logger log.Logger
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might be nice to make the logger global to the rpc package

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea.

// Listen implementation of Listen interface to listen for channel changes
func (l *StorageChangeListener) Listen() {
for change := range l.channel {
if change != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick but it might be nice to do

if change == nil {
    continue
}

// check if change key .... 

same for below

@edwardmack edwardmack merged commit 5950064 into development Jul 16, 2020
@edwardmack edwardmack deleted the ed/subscribe_storage branch July 16, 2020 14:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement State Subscribe Storage Websocket
2 participants